Learning R - Exploring the COVID Crime Effect in London
Exploring Predictors of COVID Crime Shifts
The lockdown and social distancing measures that were brought in throughout the world to tackle COVID in 2020 have had a significant, widespread effect on crime. In this notebook, I use public London crime data on robbery and burglary to examine where this “COVID crime shift” was strongest, and whether any specific drivers or correlates can be identified.
Our findings suggest that the relative change in burglary and robbery trends during “lockdown” in April and May 2020 was heavily affected by local characteristics: areas with a high residential population saw teh sharpest decreases in burglary (likely due to a reduction in available targets) while the reduction in robberies instead seem to be driven by geographic features and proxy indicators of deprivation (potentially suggesting more available targets for robbery in communities least able to work for from).
The primary purpose of this exercise was to learn R - I’ve previously worked entirely in Python, which is more than sufficient 99% of the time, but has at times proved a blocker when I want to tackle some more experimental geospatial methods or tools geared towards the academic community. With that in mind, this is likely to be a little messy, and I’ll aim to condense my main lessons into a blog post in the future. The models are not heavily tuned (aiming to explore correlates rather than provide accurate predictions) and certain predictors are likely to correlate with each other, and as such likely do not imply direct causations.
Resources I’ve used
- Matt Ashby Crime Mapping course: https://github.com/mpjashby/crimemapping/
- Spatial Modelling for Data Scientists: https://gdsl-ul.github.io/san/
- R for Data Science: https://r4ds.had.co.nz/index.html
- Geocomputation with R: https://geocompr.robinlovelace.net/
Tasks
- Ingest burglary and robbery data and assign to MSOA
- Predict trend by MSOA
- Identify COVID effect/error by MSOA
- Model
Data
In this exercise, I’m using three years of Metropolitan Police Service data from data.police.uk - I’ve downloaded these manually rather than using their API.
1. Ingest burglary and robbery data and assign to MSOA
Unlike Python has the benefit of being “focused” - it’s primaryily a tool for academics/scientists, rather than a programming language. That means the eco-system is refreshingly helpful: at first glance, there is one primary IDE, one package library, and it all works. Unlike Python, most libraries are imported globally, and that works okay…mostly
# Data manipulation, transformation and visualisation
library(tidyverse)
# Nice tables
# Simple features (a standardised way to encode vector data ie. points, lines, polygons)
# Spatial objects conversion
library(sp)
library(tmap)
# Colour palettes
library(RColorBrewer)
# More colour palettes
library(viridis)
#ggplot organiastion
library(ggpubr)
library(rgdal) # input/output, projections
library(lubridate) #date and time
#random forest and metrics
library(randomForest)require(caTools)
library(DALEX)library(dplyr)
library(corrr)
Importing data from CSV files behaves quite similarly to Python. To build our process, we’ll start by taking one month of crime data, exploring it, and writing all our steps for automation.
test_df <- read.csv("crimes/2018-01/2018-01-metropolitan-street.csv")
test_dfError in gregexpr(calltext, singleline, fixed = TRUE) :
regular expression is invalid UTF-8
Error in gregexpr(calltext, singleline, fixed = TRUE) :
regular expression is invalid UTF-8Our crime is categories according to the Home Office major crime types, and like Python, we can list them all through the “unique” function. Here I’ll be focusing on robbery and burglary: two crime types that are heavily reliant on encountering victim’s in public spaces, and as such should be affected by the “COVID effect”.
unique(test_df["Crime.type"])To avoid this getting particularly computationally intensive, let’s write a function to pull out robberies and burglaries, and assign them a specific MSOA. Then we can iterate over all our months and get monthly counts for each offence type.
subset_df <- filter(test_df, Crime.type=="Burglary" | Crime.type=="Robbery")
subset_dfOur single month of data contains 10,501 crimes.
There are R/Python quirks that will take some getting used to. While in Python, most columns can be referenced through their string name ([“name”]), R seems somewhat fussier. Other than that, the move from one to another is largely intuitive.
We now need to link this to our spatial data. We use the MSOA borders provided by MOPAC, and use the UK National Grid coordinate system. Police.uk does not use that system, so we’ll need to reproject our crime data.
lsoa_borders <- st_read("msoa_borders/MSOA_2011_London_gen_MHW.tab", crs=27700)Reading layer `MSOA_2011_London_gen_MHW' from data source `C:\Users\andre\Dropbox\Data Projects\Covid_crime_shift\msoa_borders\MSOA_2011_London_gen_MHW.tab' using driver `MapInfo File'
Simple feature collection with 983 features and 12 fields
geometry type: MULTIPOLYGON
dimension: XY
bbox: xmin: 503574.2 ymin: 155850.8 xmax: 561956.7 ymax: 200933.6
projected CRS: OSGB 1936 / British National Gridlsoa_bordersSimple feature collection with 983 features and 12 fields
geometry type: MULTIPOLYGON
dimension: XY
bbox: xmin: 503574.2 ymin: 155850.8 xmax: 561956.7 ymax: 200933.6
projected CRS: OSGB 1936 / British National Grid
First 10 features:
MSOA11CD MSOA11NM LAD11CD LAD11NM RGN11CD RGN11NM UsualRes HholdRes ComEstRes PopDen Hholds AvHholdSz
1 E02000001 City of London 001 E09000001 City of London E12000007 London 7375 7187 188 25.5 4385 1.6
2 E02000002 Barking and Dagenham 001 E09000002 Barking and Dagenham E12000007 London 6775 6724 51 31.3 2713 2.5
3 10045 10033 12 46.9 3834 2.6
4 E02000004 Barking and Dagenham 003 E09000002 Barking and Dagenham E12000007 London 6182 5937 245 24.8 2318 2.6
5 E02000005 Barking and Dagenham 004 E09000002 Barking and Dagenham E12000007 London 8562 8562 0 72.1 3183 2.7
6 8791 8672 119 50.6 3441 2.5
7 E02000008 Barking and Dagenham 007 E09000002 Barking and Dagenham E12000007 London 11569 11564 5 81.5 4591 2.5
8 E02000009 Barking and Dagenham 008 E09000002 Barking and Dagenham E12000007 London 8395 8376 19 87.4 3212 2.6
9 E02000010 Barking and Dagenham 009 E09000002 Barking and Dagenham E12000007 London 8615 8615 0 76.8 3292 2.6
10 E02000011 Barking and Dagenham 010 E09000002 Barking and Dagenham E12000007 London 6187 6086 101 38.8 2289 2.7
geometry
1 MULTIPOLYGON (((532135.1 18...
2 MULTIPOLYGON (((548881.6 19...
3 MULTIPOLYGON (((549102.4 18...
4 MULTIPOLYGON (((551550 1873...
5 MULTIPOLYGON (((549099.6 18...
6 MULTIPOLYGON (((549819.9 18...
7 MULTIPOLYGON (((548171.4 18...
8 MULTIPOLYGON (((546855 1863...
9 MULTIPOLYGON (((549618.8 18...
10 MULTIPOLYGON (((550244.1 18...Unlike our previous dataframes, this isn’t “tidy” (due to a slightly different format containing geographical data)
plot(lsoa_borders)plotting the first 9 out of 12 attributes; use max.plot = 12 to plot all
Before we can link our crimes to MSOA, we’ll need to ensure identical coordinate systems, but before we do that, we’ll need to erase any missing values.
#count missing values in the longitude column
sum(is.na(subset_df["Longitude"]))[1] 82As such, we have 82 crimes with no geographic identifiers, which we remove from our analysis.
clean_df <- subset_df[!rowSums(is.na(subset_df["Longitude"])), ]
clean_dfWe can now convert our crime data to spacial data, using our longitude and latitude coordinates - this allows us to quickly plot our data, and confirm it looks right.
subset_spatial <- st_as_sf(clean_df, coords = c("Longitude", "Latitude"),
crs = 4326, remove = FALSE)
subset_spatialSimple feature collection with 10419 features and 12 fields
geometry type: POINT
dimension: XY
bbox: xmin: -0.492381 ymin: 51.28683 xmax: 0.273434 ymax: 51.68564
geographic CRS: WGS 84
First 10 features:
Crime.ID Month Reported.by Falls.within Longitude Latitude
1 628e0d673aa1b6a70479342a64b02884499df85b18dcd63cc9bff3cff9f704bc 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
2 f8e9db16dca534a83493198a838567aa5adc9dd56496edc2fff5bb4c62b8303e 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
3 cc34822074b130f141f16d02fdb2d500c86e22ae18324b43a3231b381af3f45c 2018-01 Metropolitan Police Service Metropolitan Police Service 0.135554 51.58499
4 10de581c3cd0a8c9b970824cd7589d13148d63a70b3115d95ef6c24dc0bd2c3b 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
5 50ad5d2dfea24afec9e17218db62b3d29786775db1060634ae7d4a6e7cafc3ff 2018-01 Metropolitan Police Service Metropolitan Police Service 0.127794 51.58419
6 95abc6eb0b755c9250d19bbe0062fcd4a509b701964d89667401c9dc96ca257d 2018-01 Metropolitan Police Service Metropolitan Police Service 0.138439 51.57850
7 035cc894d732addb5009148d8e163e6360094cfe451f621348f1c0419b9cbc77 2018-01 Metropolitan Police Service Metropolitan Police Service 0.138439 51.57850
8 495cac920dcf9e0e4927074e8ac307f17d340f01c69e434c4a3721df017cd342 2018-01 Metropolitan Police Service Metropolitan Police Service 0.139479 51.57974
9 48234e70cbc22265ee7968da92df1ca72f83b45414cce486ec2203daa4e59fa2 2018-01 Metropolitan Police Service Metropolitan Police Service 0.135119 51.57849
10 3f4ba20780987c37816ff34fd0f7760cf503b2e648777f84ee0104432cb01d66 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140452 51.58110
Location LSOA.code LSOA.name Crime.type Last.outcome.category Context geometry
1 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Offender sent to prison NA POINT (0.140035 51.58911)
2 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Investigation complete; no suspect identified NA POINT (0.140035 51.58911)
3 On or near Rose Lane E01000027 Barking and Dagenham 001A Burglary Status update unavailable NA POINT (0.135554 51.58499)
4 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Status update unavailable NA POINT (0.140035 51.58911)
5 On or near Hope Close E01000028 Barking and Dagenham 001B Burglary Status update unavailable NA POINT (0.127794 51.58419)
6 On or near Geneva Gardens E01000029 Barking and Dagenham 001C Burglary Investigation complete; no suspect identified NA POINT (0.138439 51.5785)
7 On or near Geneva Gardens E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA POINT (0.138439 51.5785)
8 On or near Yew Tree Gardens E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA POINT (0.139479 51.57974)
9 On or near Portland Close E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA POINT (0.135119 51.57849)
10 On or near Pedestrian Subway E01000030 Barking and Dagenham 001D Robbery Investigation complete; no suspect identified NA POINT (0.140452 51.5811)plot(subset_spatial)plotting the first 9 out of 12 attributes; use max.plot = 12 to plot all
Success! That looks faintly promising. Now, let’s figure out how to re-project.
latlong = "+init=epsg:4326"
ukgrid = "+init=epsg:27700"
subset_osgb <- st_transform(subset_spatial, ukgrid)GDAL Message 1: +init=epsg:XXXX syntax is deprecated. It might return a CRS with a non-EPSG compliant axis order.subset_osgbSimple feature collection with 10419 features and 12 fields
geometry type: POINT
dimension: XY
bbox: xmin: 504499 ymin: 155908 xmax: 557677 ymax: 200168
projected CRS: OSGB 1936 / British National Grid
First 10 features:
Crime.ID Month Reported.by Falls.within Longitude Latitude
1 628e0d673aa1b6a70479342a64b02884499df85b18dcd63cc9bff3cff9f704bc 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
2 f8e9db16dca534a83493198a838567aa5adc9dd56496edc2fff5bb4c62b8303e 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
3 cc34822074b130f141f16d02fdb2d500c86e22ae18324b43a3231b381af3f45c 2018-01 Metropolitan Police Service Metropolitan Police Service 0.135554 51.58499
4 10de581c3cd0a8c9b970824cd7589d13148d63a70b3115d95ef6c24dc0bd2c3b 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
6 95abc6eb0b755c9250d19bbe0062fcd4a509b701964d89667401c9dc96ca257d 2018-01 Metropolitan Police Service Metropolitan Police Service 0.138439 51.57850
7 035cc894d732addb5009148d8e163e6360094cfe451f621348f1c0419b9cbc77 2018-01 Metropolitan Police Service Metropolitan Police Service 0.138439 51.57850
8 495cac920dcf9e0e4927074e8ac307f17d340f01c69e434c4a3721df017cd342 2018-01 Metropolitan Police Service Metropolitan Police Service 0.139479 51.57974
9 48234e70cbc22265ee7968da92df1ca72f83b45414cce486ec2203daa4e59fa2 2018-01 Metropolitan Police Service Metropolitan Police Service 0.135119 51.57849
10 3f4ba20780987c37816ff34fd0f7760cf503b2e648777f84ee0104432cb01d66 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140452 51.58110
Location LSOA.code LSOA.name Crime.type Last.outcome.category Context geometry
1 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Offender sent to prison NA POINT (548349 189976)
2 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Investigation complete; no suspect identified NA POINT (548349 189976)
3 On or near Rose Lane E01000027 Barking and Dagenham 001A Burglary Status update unavailable NA POINT (548052 189507.9)
4 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Status update unavailable NA POINT (548349 189976)
5 On or near Hope Close E01000028 Barking and Dagenham 001B Burglary Status update unavailable NA POINT (547517 189404)
6 On or near Geneva Gardens E01000029 Barking and Dagenham 001C Burglary Investigation complete; no suspect identified NA POINT (548273 188793)
7 On or near Geneva Gardens E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA POINT (548273 188793)
8 On or near Yew Tree Gardens E01000029 Barking and Dagenham 001C Burglary Status update unavailable Burglary Status update unavailable NA POINT (548043 188785)
10 On or near Pedestrian Subway E01000030 Barking and Dagenham 001D Robbery Investigation complete; no suspect identified NA POINT (548404 189086)We now run a “spatial join”: assigning an MSOA to each of our crimes, based on the MSOA border map they are within, and combining these as one table.
crime_with_msoa <- st_join(subset_osgb, lsoa_borders["MSOA11CD"])
crime_with_msoaSimple feature collection with 10419 features and 13 fields
geometry type: POINT
dimension: XY
bbox: xmin: 504499 ymin: 155908 xmax: 557677 ymax: 200168
projected CRS: OSGB 1936 / British National Grid
First 10 features:
Crime.ID Month Reported.by Falls.within Longitude Latitude
1 628e0d673aa1b6a70479342a64b02884499df85b18dcd63cc9bff3cff9f704bc 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
2 f8e9db16dca534a83493198a838567aa5adc9dd56496edc2fff5bb4c62b8303e 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
3 cc34822074b130f141f16d02fdb2d500c86e22ae18324b43a3231b381af3f45c 2018-01 Metropolitan Police Service Metropolitan Police Service 0.135554 51.58499
4 10de581c3cd0a8c9b970824cd7589d13148d63a70b3115d95ef6c24dc0bd2c3b 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140035 51.58911
5 50ad5d2dfea24afec9e17218db62b3d29786775db1060634ae7d4a6e7cafc3ff 2018-01 Metropolitan Police Service Metropolitan Police Service
6 95abc6eb0b755c9250d19bbe0062fcd4a509b701964d89667401c9dc96ca257d 2018-01 Metropolitan Police Service Metropolitan Police Service 0.138439 51.57850
7 035cc894d732addb5009148d8e163e6360094cfe451f621348f1c0419b9cbc77 2018-01 Metropolitan Police Service Metropolitan Police Service 0.138439 51.57850
8 495cac920dcf9e0e4927074e8ac307f17d340f01c69e434c4a3721df017cd342 2018-01 Metropolitan Police Service Metropolitan Police Service 0.139479 51.57974
9 48234e70cbc22265ee7968da92df1ca72f83b45414cce486ec2203daa4e59fa2 2018-01 Metropolitan Police Service Metropolitan Police Service 0.135119 51.57849
10 3f4ba20780987c37816ff34fd0f7760cf503b2e648777f84ee0104432cb01d66 2018-01 Metropolitan Police Service Metropolitan Police Service 0.140452 51.58110
Location LSOA.code LSOA.name Crime.type Last.outcome.category Context MSOA11CD
1 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Offender sent to prison NA E02000002
2 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Investigation complete; no suspect identified NA E02000002
3 On or near Rose Lane E01000027 Barking and Dagenham 001A Burglary Status update unavailable NA E02000002
4 On or near Beansland Grove E01000027 Barking and Dagenham 001A Burglary Status update unavailable NA E02000002
5 On or near Hope Close E01000028 Barking and Dagenham 001B Burglary Status update unavailable NA E02000002
6 On or near Geneva Gardens E01000029 Barking and Dagenham 001C Burglary Investigation complete; no suspect identified NA E02000002
7 On or near Geneva Gardens E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA E02000002
8 On or near Yew Tree Gardens E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA E02000002
9 On or near Portland Close E01000029 Barking and Dagenham 001C Burglary Status update unavailable NA E02000002
10 On or near Pedestrian Subway E01000030 Barking and Dagenham 001D Robbery Investigation complete; no suspect identified NA E02000002
geometry
1 POINT (548349 189976)
2 POINT (548349 189976)
3 POINT (548052 189507.9)
4 POINT (548349 189976)
5 POINT (547517 189404)
6 POINT (548273 188793)
7 POINT (548273 188793)
8 POINT (548341 188933)
9 POINT (548043 188785)
10 POINT (548404 189086)That has worked: each crime is now assigned to an MSOA. We can now group our table by count of MSOA, to obtain the monthly count of
msoa_list<- crime_with_msoa %>%
group_by(MSOA11CD, Crime.type) %>%
summarize(count_by_msoa = n())`summarise()` has grouped output by 'MSOA11CD'. You can override using the `.groups` argument.msoa_listSimple feature collection with 1722 features and 3 fields
geometry type: GEOMETRY
xmin: 504499 ymin: 155908 xmax: 557677 ymax: 200168
projected CRS: OSGB 1936 / British National GridWe now have a count for each, and we can remove the geometry data before automating our process.
msoa_pivot_tibble <- as_tibble(msoa_list)
msoa_pivot_tibble <- msoa_pivot_tibble[0:3]
msoa_pivot_tibbleNAAs a final stage prior to processing the rest of our data, we will fill any missing msoa/month combinations with a “0”, to ensure consistent trends.
#creating a df with all msoa names, for robbery and burglary
msoa_zero_df_robbery <- unique(as_tibble(lsoa_borders)["MSOA11CD"])
msoa_zero_df_burglary <- unique(as_tibble(lsoa_borders)["MSOA11CD"])
#adding our crime type column
msoa_zero_df_burglary["Crime.type"] = "Burglary"
msoa_zero_df_robbery["Crime.type"] = "Robbery"
#Creating a "count" column identical to our pivot, and filling it with 0
msoa_zero_df_robbery["count_by_msoa"] = as.numeric(0)
#combine both
duplicate_concat <- rbind(msoa_zero_df_robbery, msoa_zero_df_burglary)
#add our duplicates to our original table
df_with_dups <- rbind(msoa_pivot_tibble, duplicate_concat)
dup_filters <- duplicated(df_with_dups[0:2])
#bring it all back together
monthly_df <- filter(df_with_dups, !dup_filters)
monthly_df
#select the first unique value of months in the original dataframe
month <- unique(test_df["Month"])[1,1]
monthly_df["Month"] <- month
monthly_dfNAwe now have our basic functionality to create a time series for both crime types, by MSOA - a count of each type, and the month.
I can now bring all my previous work together into a function, which we can use to automate the process.
#quick initial function to generate our MSOA borde spatial frame, to avoid it sitting in the initial frame and gobbling loads of memory.
generate_msoa_borders <- function(file){
msoa_borders <- st_read(file, crs=27700)
return(msoa_borders)
}
make_month_pivot <- function(file){
#define our CRS
latlong = "+init=epsg:4326"
ukgrid = "+init=epsg:27700"
#read our crime from the file
test_df <- read.csv(file)
#select only our target crime types
subset_df <- filter(test_df, Crime.type=="Burglary" | Crime.type=="Robbery")
#remove any rows with a long/lat coordinate
clean_df <- subset_df[!rowSums(is.na(subset_df["Longitude"])), ]
#generate a spatial df
subset_spatial <- st_as_sf(clean_df, coords = c("Longitude", "Latitude"),
crs = 4326, remove = FALSE)
#reproject to uk grid coords
subset_osgb <- st_transform(subset_spatial, ukgrid)
#spatially join to assign to an MSOA
crime_with_msoa <- st_join(subset_osgb, msoa_borders["MSOA11CD"])
#summarise by count of MSOA
msoa_list<- crime_with_msoa %>%
group_by(MSOA11CD, Crime.type) %>%
#return to a non-geographic msoa
msoa_pivot_tibble <- as_tibble(msoa_list)
msoa_pivot_tibble <- msoa_pivot_tibble[0:3]
#creating a df with all msoa names, for robbery and burglary
msoa_zero_df_robbery <- unique(as_tibble(msoa_borders)["MSOA11CD"])
msoa_zero_df_burglary <- unique(as_tibble(msoa_borders)["MSOA11CD"])
msoa_zero_df_burglary["Crime.type"] = "Burglary"
msoa_zero_df_robbery["Crime.type"] = "Robbery"
#Creating a "count" column identical to our pivot, and filling it with 0
msoa_zero_df_burglary["count_by_msoa"] = as.numeric(0)
duplicate_concat <- rbind(msoa_zero_df_robbery, msoa_zero_df_burglary)
df_with_dups <- rbind(msoa_pivot_tibble, duplicate_concat)
#creating a filter for duplicates columns, which should ignore the first instance
dup_filters <- duplicated(df_with_dups[0:2])
monthly_df <- filter(df_with_dups, !dup_filters)
month <- unique(test_df["Month"])[1,1]
monthly_df["Month"] <- month
return(monthly_df)
}We now have a rudimentary tooling pipeline, which we can iterate over our subfolders to aggregate our data (in hindset, I probably should have explored the API options).
#create empty dataframe to bring together our data
empty_df <- tibble(
MSOA11CD = "",
Crime.type= "",
count_by_msoa= "",
Month= ""
)
#re-ingest our MSOA data
msoa_borders <- generate_msoa_borders("msoa_borders/MSOA_2011_London_gen_MHW.tab")Our function will iterate over each file, add them to our empty table, until each is complete (and we have a single, aggregated file)
subfiles <- list.files(path = "crimes", recursive=T)
for (file in subfiles){
folder_subdir <- "crimes/"
#concatenate to get our total subfolder directory - hacky but will work here.
sub_path <- paste(folder_subdir, file, sep="")
monthly_df <- make_month_pivot(sub_path)
}empty_dfWe now have a combined dataframe of 71,848 rows, from January 2018 through December 2020.
unique(empty_df["Month"])From a practical perspective, that was slightly more painful than I expected - I’m not sure if data-wrangling is less intuitive in R, or it’s just practice, but it’s noticeable how easy use cases are less easily accessible as tutorials (probably due to the different user base )
#saving file to CSV
#write.csv(empty_df,"msoa_crime_matrix.csv")
2. Predict trend by MSOA
Visualisation and Exploration
With our data now cleaned and aggregated, we can focus on the more interesting part - forecasting our “expected” pandemic crime, and examining how much it diverges from our “actual” crime.
empty_df <- read.csv("msoa_crime_matrix.csv")
empty_df <- empty_df[2:70848,2:5]
empty_dfBefore going any further, let’s use this to explore and visualise the distribution of robbery and burglary across time and space during our “pre-pandemic” period, in March 2020 - based on London mobility indicators, this is when movement accross London began to be heavily affected, and the disruption was most notable in April
London mobility data
burglary_df<-empty_df
#add a "1" so our month can be converted to a full date
burglary_df$DateString <- paste(burglary_df$Month, "-01", sep="")
#convert to date format
burglary_df$DateClean <- ymd(burglary_df$DateString)
#filter out only burglary prior to the pandemic
burglaryExplore <- filter(burglary_df, DateClean < "2020-03-01" & Crime.type=="Burglary")
burglaryExploreLooking at the aggregate counts of burglary across London, a visual observation suggests yearly trends (which we’ll have to consider in our forecast), which sharp peaks during the Winter months and the lowest numbers in summer (when the days are longest).
burglary_by_month <- burglaryExplore %>%
group_by(DateClean) %>%
summarize(total_burglaries = sum(count_by_msoa))
ggplot(burglary_by_month, aes(x=DateClean, y=total_burglaries)) +
geom_line()
To observe how crime counts are distributed in space, let’s map both counts by MSOA. As previously mentioned, MSOAs are designed to be comparable units, at least from a population perspective - we don’t need to produce per population rates.
burglary_by_msoa <- burglaryExplore %>%
group_by(MSOA11CD) %>%
summarize(total_burglaries = sum(count_by_msoa))
burglary_map <- left_join(lsoa_borders, burglary_by_msoa, by = "MSOA11CD")
pal <- brewer.pal(5,"BuGn")
tm_fill(col = "total_burglaries", title = "Total Burglary Count by MSOA", style="quantile", palette="BuGn") +
tm_layout(legend.outside = TRUE, legend.outside.position = "right")
robbery_df<-empty_df
robbery_df$DateString <- paste(robbery_df$Month, "-01", sep="")
robbery_df$DateClean <- ymd(robbery_df$DateString)
robberyExplore <- filter(robbery_df, DateClean < "2020-03-01" & Crime.type=="Robbery")
robbery_by_msoa <- robberyExplore %>%
group_by(MSOA11CD) %>%
summarize(total_robberies = sum(count_by_msoa))
robbery_map <- left_join(lsoa_borders, robbery_by_msoa, by = "MSOA11CD")
pal <- brewer.pal(5,"BuGn")
robbery_map <-tm_shape(robbery_map) +
tm_fill(col = "total_robberies", title = "Total Robbery Count by MSOA", style="quantile", palette="BuGn") +
tmap_arrange(burglary_map, robbery_map, nrow = 2)
We notice that robbery is noticeably more concentrated in central London, with burglary remaining quite common across the city. That said, there are also obvious spatial patterns here - these crimes are clustered in certain geographies.
Modelling
We can now begin the forecasting process. To design our process, we’ll start by focusing on a single MSOA - the first in our dataset, E02000001, or the City of London.
single_msoa_df <- filter(empty_df, MSOA11CD == "E02000001" & Crime.type=="Burglary")
single_msoa_df$DateString <- paste(single_msoa_df$Month, "-01")
single_msoa_dfFrom a forecasting/time-series perspective, this is a very small dataset - 36 monthly observations. We will be shrinking this further to only 26 by focusing on data prior to March 2020, when the COVID crime impact is felt. This significantly limits our forecasting options, and will impact accuracy, if we treat each MSOA in isolation - we could explore some sort of Vector Autoregressive Model to limit this, but given that we’re then going to be exploring the error of all our models in aggregation, this isn’t crucial. Our focus is on models that we can accurately deploy without needing to tune each of them individually, and that can capture the seasonal trend, and generate reliable predictions on our limited dataset.
Given these limitations, I’ve opted for the Prophet algorith. While it’s more opaque than a auto-arima or VAR model, it works well with monthly data, and extracting seasonal trends. It also requires very little tuning.
As such, we’ll extract our “training set” prior to March, and start forecasting.
training_set <- filter(single_msoa_df, DateClean < "2020-03-01")
training_df <- tibble(
y=training_set$count_by_msoa
)
training_dflibrary(prophet)
m <- prophet(training_df)For now, we’ll forecast on a 6 month horizon - we obviously wouldn’t expect it to be accurate that far into the future.
future <- make_future_dataframe(m, periods = 6, freq = 'month')
forecast <- predict(m, future)
plot(m, forecast)
As we can see, the model seems consistent on a short horizon, and gets very wide as it goes further into the future. More importantly however, it has extracted a yearly seasonal compontent - the summer decrease we identified previously - as well as a long term trend.
prophet_plot_components(m, forecast)
These predictions seem far-fetched, but remember we will be observing a London wide error rate. As such, we must now isolate our “pandemic period” - which we define as April and May 2020 - and compare the predicted crime counts to the actual crime counts to obtain a metric of our “COVID crime shift”, or our error rate.
forecast$Month <- month(forecast$ds)
forecast$Year <- year(forecast$ds)
this_year <- filter(forecast, Year > 2019)
peak_pandemic <- filter(this_year, Month== 4 | Month== 5 )
predictionPivot <- peak_pandemic %>%
group_by(Month) %>%
single_msoa_df$MonthNum <- month(single_msoa_df$DateClean)
single_msoa_df$YearNum <- year(single_msoa_df$DateClean)
this_year_actual <- filter(single_msoa_df, YearNum > 2019)
peak_pandemic_actual <- filter(this_year_actual, MonthNum== 4 | MonthNum== 5 )
actual_burglary <- sum(peak_pandemic_actual$count_by_msoa)
pred_burglary <- sum(predictionPivot$predicted_burglary)
error <- actual_burglary - pred_burglary
percentage_error <- error / pred_burglary
print("Burglary Count")[1] "Burglary Count"print(actual_burglary)
print("Predicted")[1] "Predicted"print(pred_burglary)[1] 7.625019print("Actual Error")print(error)[1] -6.625019print("Percentage Error")[1]print(percentage_error)[1] -0.8688528In this MSOA, our model predicted nearly 8 burglaries would occur in these two months, based on pre-pandemic trends. In reality, 1 took place - a large error rate, suggesting a strong “COVID effect”.
This process can now be replicated for every MSOA in London, to obtain this metric for each MSOA.
msoa_error_tibbleOur process has completed: we have a “COVID shift” measure for all of London.
3. Measuring Local COVID Crime Shifts
We now need to use our forecasts to measure the “error” - this should provide an indication of the “COVID Crime Shift”, or how much the actual crime diverted from the previous forecasts.
I explored various avenues for this: the ideal solution would be a relative rate of the error, as MSOAs with large crime numbers will likely generate large errors, and so a rate would be ideal, though this is complicated by our erratic prediction and mix of positive and negative numbers.
Our final solution has explored two options: - the absolute error number - the relative error once the crime and predictions have been transformed (by adding 50)
We visualise and describe these statistics first to ensure they appear sensible.
#write_csv(msoa_error_tibble, "msoa_error_table2.csv")msoa_error_tibble <- read_csv("msoa_error_table2.csv")
#msoa_error_tibble<- msoa_error_table[2:980,]
msoa_error_tibble <- msoa_error_tibble[2:980, ]
msoa_error_tibble <- left_join(msoa_error_tibble, robbery_by_msoa, by = "MSOA11CD")
msoa_error_tibble <- left_join(msoa_error_tibble, burglary_by_msoa, by = "MSOA11CD")
msoa_error_tibble$RPDRobbery <- (msoa_error_tibble$robberyActual - msoa_error_tibble$robberyPredicted)/((msoa_error_tibble$robberyPredicted + msoa_error_tibble$robberyActual)/2)
msoa_error_tibble$robberyActualShifted <- msoa_error_tibble$robberyActual + 50
msoa_error_tibble$robberyPredictedShifted <- msoa_error_tibble$robberyPredicted + 50
msoa_error_tibble$burglaryActualShifted <- msoa_error_tibble$burglaryActual + 50
msoa_error_tibble$burglaryPredictedShifted <- msoa_error_tibble$burglaryPredicted + 50
print("Burglary Error")[1] "Burglary Error"
Min. 1st Qu. Median Mean 3rd Qu. Max.
-97.191 -12.870 -5.271 -6.090 2.111 48.727 print("Burglary Relative Error")[1] "Burglary Relative Error"summary(msoa_error_tibble$RPDburglaryShifted) Min. 1st Qu. Median Mean 3rd Qu. Max.
-0.81266 -0.20689 -0.08826 -0.07976 0.03612 1.29467
[1] "Robbery Error"summary(msoa_error_tibble$robberyError) Min. 1st Qu. Median Mean 3rd Qu. Max.
-2.150 -3.840 2.205 25.564 print("Robbery Relative Error")[1] "Robbery Relative Error"summary(msoa_error_tibble$RPDRobberyShifted) Min. 1st Qu. Median Mean 3rd Qu. Max.
-1.45491 -0.11808 -0.04081 -0.04876 0.04309 0.62020 As we can see, the average London MSOA experienced a negative COVID crime shift for both burglary and robbery, but this is far from equally distributed - at the extremes, some areas actually see large increases on our predicted values.
library(ggpubr)Loading required package: ggplot2
Registered S3 method overwritten by 'data.table':
method from
print.data.table burg_hist <- ggplot(msoa_error_tibble, aes(x=burglaryError)) + geom_histogram()
rob_hist <-ggplot(msoa_error_tibble, aes(x=robberyError)) + geom_histogram()
burg_r_hist <- ggplot(msoa_error_tibble, aes(x=RPDburglaryShifted)) + geom_histogram()
rob_r_hist <- ggplot(msoa_error_tibble, aes(x=RPDRobberyShifted)) + geom_histogram()
scatter <- ggplot(msoa_error_tibble, aes(x = RPDRobberyShifted, y = RPDburglaryShifted)) +
geom_point()
r_scatter <- ggplot(msoa_error_tibble, aes(x = robberyError, y = burglaryError)) +
geom_point()
ggarrange(rob_hist, burg_hist, rob_r_hist, burg_r_hist,scatter, r_scatter, ncol=2, nrow=3 )`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
`stat_bin()` using `bins = 30`. Pick better value with `binwidth`.
Our shifted relative error rate seems to function as intended: while there are still outliers, they are more concentrated than they are for the pure error term, and the overall distribution is more focused, while still indicating the direction and relative strength of our COVID effect.
Let’s map this effect visually, and see if any particular areas stand out.
#re-ingest our geographic MSOA borders
msoa_borders <- st_read("msoa_borders/MSOA_2011_London_gen_MHW.tab", crs=27700)Reading layer `MSOA_2011_London_gen_MHW' from data source `' using driver `MapInfo File'
Simple feature collection with 983 features and 12 fields
geometry type: MULTIPOLYGON
dimension: XY
bbox: xmin: 503574.2 ymin: 155850.8 xmax: 561956.7 ymax: 200933.6
projected CRS: OSGB 1936 / British National Gridgeographic_error_map <- left_join(msoa_borders, msoa_error_tibble, by = "MSOA11CD")
burg_map <- tm_shape(geographic_error_map) +
tm_layout(legend.outside = TRUE, legend.outside.position = "right")
rob_map <-tm_shape(geographic_error_map) +
tm_layout(legend.outside = TRUE, legend.outside.position = "right")
burg_map_rate <- tm_shape(geographic_error_map) +
tm_fill(col = "RPDRobberyShifted", title = "Robbery Error Relative", palette="-PuOr")+
tm_layout(legend.outside = TRUE, legend.outside.position = "right")
rob_map_rate <-tm_shape(geographic_error_map) +
tmap_arrange(burg_map, rob_map, burg_map_rate, rob_map_rate , nrow = 2, ncol=2)
It’s hard to identify any obvious effect visually, but we do notice that while central London sees some very strong reductions, it also sees some increases. Conversely, the outskirts of London (notably to the south and West) are a near continuous area of large decreases. The effect does vary by offence type, but the pattern seen in South and West London appears broadly consistent.
4. Identifying Correlates and Modelling
We’ve identified that the COVID crime effect was felt unequally accross London, and varies by offence type. To finalise our project, we will be linking our data to demographic data provided by MOPAC, and aiming to use it to identify correlates to our “covid shift”, and hopefully build models disentangling the effect.
library(readxl)
#ingest ATLAS
msoa_atlas <- read_excel("msoa_atlas/msoa-data.xls")New names:
* `House Prices Sales 2011` -> `House Prices Sales 2011...129`
* `House Prices Sales 2011` -> `House Prices Sales 2011...130`geographic_msoa_matrix <- left_join(geographic_error_map, msoa_atlas, by = "MSOA11CD")
#convert to tibble
msoa_matrix_tbl <- as_tibble(geographic_msoa_matrix)
#select only numeric data
msoa_matrix_numeric <-dplyr::select_if(msoa_matrix_tbl, is.numeric)
msoa_matrix_numericHaving now ingested and linked our data, we begin by exploring the factors most highly correlated with our relative error rates.
corr_df <- correlate(dplyr::select_if(msoa_matrix_tbl, is.numeric), quiet = TRUE)
options(scipen=999)Starting by our relative robbery error, a few interesting correlates stand out: - road traffic casualties - burglary numbers - the number of dwellings with no usual residents, and the number of commercial residents - households with no cars - the age composition of the area - general deprivation indicators (such as the proportion of households with central heating)
#show only correlates with an absolute value higher than 0.2
#filter(dplyr::select(corr_df[order(corr_df$RPDRobberyShifted),] , term, RPDRobberyShifted), RPDRobberyShifted < -0.2 | RPDRobberyShifted > 0.2)
#filter(dplyr::select(corr_df[order(corr_df$RPDRobberyShifted),] , term, RPDRobberyShifted), RPDRobberyShifted < -0.2 | RPDRobberyShifted > 0.2)
high_corr_rob <- filter(dplyr::select(corr_df, term, RPDRobberyShifted), RPDRobberyShifted < -0.15 | RPDRobberyShifted > 0.15)
high_corr_rob[order(high_corr_rob$RPDRobberyShifted),]NAThe correlations for burglary are weaker - only a few have an absolute value higher than 0.2 - but a few stand out: - households with no residents - high robbery numbers - house prices
high_corr_burg <- filter(dplyr::select(corr_df, term, RPDburglaryShifted), RPDburglaryShifted < -0.15 | RPDburglaryShifted > 0.15)
high_corr_burg[order(high_corr_burg$RPDburglaryShifted),]NAThese correlates suggest we can model this shift - this is likely to prove more reliable for robbery (where the correlations are stronger), and seem linked to usual resident population(as measured by household composition), deprivation (through various proxy indicators such as central heating presence or housing type), and general crime patterns (through total burglary and robbery numbers)
We will take two approaches for modelling: a simple regression (to identify strong links) and random forest regressors (to identify non-linear associations)
Simple Regression
We begin through the use of simple OLS regression. This is a linear model that has large limitations for modelling complex relationships, but can be an effective first step, effectively with a few transformations.
R does not cope well with blank spaces in terms, so we’ll extract and rename our key correlates.
msoa_copy <- msoa_matrix_numeric
names(msoa_copy)[names(msoa_copy) == "Dwelling type (2011) Household spaces with no usual residents"] <- "DwellingNoResidents"
names(msoa_copy)[names(msoa_copy) == "House Prices Median House Price (£) 2010"] <- "MedianHousePrice"
names(msoa_copy)[names(msoa_copy) == "Dwelling type (2011) Flat, maisonette or apartment"] <- "FlatAprt"
names(msoa_copy)[names(msoa_copy) == "Qualifications (2011 Census) Schoolchildren and full-time students: Age 18 and over"] <- "fullTimeStudents"
names(msoa_copy)[names(msoa_copy) == "Car or van availability (2011 Census) No cars or vans in household"] <- "NoCars"
names(msoa_copy)[names(msoa_copy) == "Ethnic Group (2011 Census) Other ethnic group"] <- "OtherEthnicGroup"
names(msoa_copy)[names(msoa_copy) == "Central Heating (2011 Census) Households with central heating (%)"] <- "CentralHealingPercent"
Let’s now extract these to a separate dataframe, remove any missing values, and provide some quick summary statistics to identify any obvious concerns.
feature_df <- dplyr::select(msoa_copy, RPDburglaryShifted, RPDRobberyShifted, total_burglaries, total_robberies, DwellingNoResidents, MedianHousePrice, FlatAprt, fullTimeStudents, NoCars, OtherEthnicGroup, CentralHealingPercent, AvHholdSz, ComEstRes)
feature_df <- drop_na(feature_df, RPDburglaryShifted, RPDRobberyShifted)
feature_dfsummary(feature_df) RPDburglaryShifted RPDRobberyShifted total_burglaries total_robberies DwellingNoResidents FlatAprt fullTimeStudents NoCars
Min. :-0.81266 Min. :-1.45491 Min. : 46.0 Min. : 2.00 Min. : 14.0 Min. : 137625 Min. : 54.0 Min. : 122.0 Min. : 185
1st Qu.:-0.20689 1st Qu.:-0.11808 1st Qu.: 121.0 1st Qu.: 29.00 1st Qu.: 59.0 1st Qu.: 222500 1st Qu.: 830.5 1st Qu.: 305.5 1st Qu.: 796
Median :-0.08826 Median :-0.04081 Median : 159.0 Median : 53.00 Median : 86.0 Median : 465.0 Median :1256
Mean :-0.07976 Mean :-0.04876 Mean : 175.5 Mean : 78.53 Mean : 123.4 Mean : 310710 Mean :1798.2 Mean : 539.5 Mean :1382
3rd Qu.: 0.03612 3rd Qu.: 0.04309 3rd Qu.: 205.0 3rd Qu.: 92.00 3rd Qu.: 133.0 3rd Qu.: 351525 3rd Qu.:2566.5 3rd Qu.: 654.5 3rd Qu.:1858
Max. : 1.29467 Max. : 0.62020 Max. :1973.0 Max. :2456.00 Max. :1556.0 Max. :1425000 Max. :6429.0 Max. :3370.0 Max. :4319
OtherEthnicGroup CentralHealingPercent AvHholdSz ComEstRes
Min. : 16.0 Min. :92.10 Min. :1.600 Min. : 0
1st Qu.: 134.0 1st Qu.:96.60 1st Qu.:2.300 1st Qu.: 9
Median : 232.0 Median :97.40 Median :2.500 Median : 41
Mean : 285.8 Mean :97.24 Mean :2.506 Mean : 102
3rd Qu.: 376.5 3rd Qu.:98.00 3rd Qu.:2.700 3rd Qu.: 105
Max. :3001.0 Max. :99.60 Max. :3.900 Max. :2172
colSums(is.na(feature_df)) RPDburglaryShifted RPDRobberyShifted total_burglaries total_robberies DwellingNoResidents MedianHousePrice FlatAprt
0 0 0 0 0 0 0
fullTimeStudents NoCars OtherEthnicGroup CentralHealingPercent AvHholdSz ComEstRes
0 0 0 0 0 0 As a quick initial exercise, we’ll create a scatter plot of the interaction between each and every one of our variables. This will probably be too messy to make any real findings, but can serve to quickly highlight strong associations.
pairs(feature_df)
As we hoped, some obvious relationships stand out: for instance, the presence of apartments, and households with no cars, or central heating and average household size.
Our robbery and burglary data and change rates are densely clustered - they’re unlikely to cleanly associate with anything. With that in mind, we’ll perform a log transformation. This cannot be undertaken with negative values, so once again we’ll perform a shift (of 2) for both of our relative error numbers, as well as a commercial resident column, before log transforming our features.
feature_df$BurglaryRPDTranform <- feature_df$RPDburglaryShifted + 2
feature_df$RobberyRPDTranform <- feature_df$RPDRobberyShifted + 2
feature_df$ComEstResTranform <- feature_df$ComEstRes + 2
for (col in colnames(feature_df)){
new_name <- paste("log_", col, sep = "")
feature_df[new_name] <- log(feature_df[col])
}NaNs producedNaNs produced
drop<- c( "log_ComEstRes", "log_RPDburglaryShifted", "log_RPDRobberyShifted")
feature_df<- feature_df[,!(names(feature_df) %in% drop)]
feat_transform_df <- feature_df[,17:29]
orig_feat_df <- feature_df[,0:17]
#feat_transform_df <- feature_df[,17:28]
colSums(is.na(feature_df)) RPDburglaryShifted RPDRobberyShifted total_burglaries total_robberies DwellingNoResidents MedianHousePrice
0 0 0 0 0 0
FlatAprt fullTimeStudents NoCars OtherEthnicGroup CentralHealingPercent AvHholdSz
0 0 0 0 0 0
ComEstRes BurglaryRPDTranform RobberyRPDTranform ComEstResTranform log_total_burglaries log_total_robberies
0 0 0 0 0 0
log_DwellingNoResidents log_MedianHousePrice log_FlatAprt log_fullTimeStudents log_NoCars log_OtherEthnicGroup
0 0 0 0 0 0
log_CentralHealingPercent log_AvHholdSz log_BurglaryRPDTranform log_RobberyRPDTranform log_ComEstResTranform
0 0 0 0 0 We’ve now separated a separate dataframe where each value has been log transformed - while this isn’t hugely rigorous (and would benefit from inspecting the relationships in more detail) it serves our immediate purpose.
pairs(feat_transform_df)
While we’ve introduced a bit of noise, we’ve also “forced” some of our variables into relationships that look semi linear.
To dig into this deeper, let’s create a correlation matrix for our entire transformed dataframe.
correlate(feat_transform_df)
Correlation method: 'pearson'
Missing treated using: 'pairwise.complete.obs'To provide a visual aid, I’ve extracted the column for our robbery relative change rate, and sorted the table accordingly.
dplyr::select(correlate(feat_transform_df)[order(correlate(feat_transform_df)$log_BurglaryRPDTranform),], term, log_BurglaryRPDTranform)
Correlation method: 'pearson'
Missing treated using: 'pairwise.complete.obs'
Correlation method: 'pearson'
Missing treated using: 'pairwise.complete.obs'Regression
Now that we’ve transformed our data, cleaned it up, and identified potential correlates, let’s build our linear model.
There are various automated tools for this process that seek to provide the highest fit and significance, but given the high degree of correlation between my chosen features, I’ve taken a more manual approach and tested a variety of models until I identified one with a suitable fit. The final model is below.
mod_burglary <- lm(log_BurglaryRPDTranform ~ log_total_burglaries + log_FlatAprt + log_MedianHousePrice + log_ComEstResTranform, data = feat_transform_df)
summary(mod_burglary)
Call:
lm(formula = log_BurglaryRPDTranform ~ log_total_burglaries +
log_FlatAprt + log_MedianHousePrice + log_ComEstResTranform,
data = feat_transform_df)
Residuals:
Min 1Q Median -0.06547 -0.00309 0.05830 0.56322
Coefficients:
(Intercept) 1.255068 0.112681 11.138 -6.062 1.92e-09 ***
log_FlatAprt 0.016153 0.005157 3.132 0.001786 ***
log_ComEstResTranform -0.008125 0.002119 -3.833 0.000135 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.1027 on 974 degrees of freedom
Multiple R-squared:
F-statistic: 21.72 974 DF, p-value: < 2.2e-16Our model suggests the largest burglary decrease linked to lockdown was in MSOAs which a high level of historic burglary. The composition of housing/accomodation and area type also seems to play a role, with those areas with higher median house prices, and a larger number of commercial residents, seeing stronger decreases, while conversely areas with large number of apartments temper the effect.
While all our variables are significant, the model is not a particularly good fit - the adjusted R2 is around 0.08, suggesting that less than 10% of the variance is accounted for by our model. I suspect more geographic features - such as distance from central London, more accurate footfall, or spatial lags - would probably be useful, but that’s outside the scope of this project.
Let’s perform a similar exercise for robbery.
dplyr::select(correlate(feat_transform_df)[order(correlate(feat_transform_df)$log_RobberyRPDTranform),], term, log_RobberyRPDTranform)
Correlation method: 'pearson'
Missing treated using: 'pairwise.complete.obs'
Correlation method: 'pearson'
Missing treated using: 'pairwise.complete.obs'mod_burglary <- lm(log_RobberyRPDTranform ~ log_total_robberies + log_total_burglaries + log_FlatAprt + log_CentralHealingPercent, data = feat_transform_df)
summary(mod_burglary)
Call:
lm(formula = log_RobberyRPDTranform ~ log_total_robberies + log_total_burglaries +
log_FlatAprt + log_CentralHealingPercent, data = feat_transform_df)
Residuals:
Min 1Q Median 3Q Max
-0.95347 -0.04269 -0.00516 0.04607 0.31963
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) -5.057332 1.394750 -3.626 0.000303 ***
log_total_robberies -0.031452 0.004902 -6.416 0.0000000002177 ***
log_total_burglaries -0.066422 0.009692 -6.853 0.0000000000128 ***
log_FlatAprt 0.029271 0.005070 5.773 0.0000000104411 ***
log_CentralHealingPercent 1.304434 0.301621 4.325 0.0000168425948 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.08917 on 974 degrees of freedom
Multiple R-squared: 0.21020.207
F-statistic: 64.81 on 4 and 974 DF, p-value: < 0.00000000000000022This is a notably better fit than our burglary model, with our R2 suggesting we now account for over 20% of the variance. The nature of our predictors is also quite different: while we still see a negative relationship with historic crime (with areas of high historic crime experiencing larger relative decreases), there is a positive relationship with both the presence of apartments and central heating.
I suspect some of these features are correlates of deprivation, so I want to create a quick scatter of three - for now we’ll do it against median house price, which is definitely deprivation correlated.
heating <- ggplot(feature_df, aes(x = log_MedianHousePrice, y = log_CentralHealingPercent)) +
geom_point()+
geom_smooth(method=lm)
apartments <- ggplot(feature_df, aes(x = log_MedianHousePrice, y = log_FlatAprt)) +
geom_smooth(method=lm)
comest <- ggplot(feature_df, aes(x = log_MedianHousePrice, y = log_ComEstResTranform)) +
geom_point()+
geom_smooth(method=lm)
`geom_smooth()` using formula 'y ~ x'
`geom_smooth()` using formula 'y ~ x'
While there does appear to be a relationship with some of these, it isn’t strong - this suggests the factor’s we have identified are significant not because of their association with deprivation and poverty, but because of what they mean about the specific characteristics of the area.
Random Forests
Unlike regression, random forest doesn’t really on any specific type of association - instead, we rely on computing power, repetition and iteration to capture the very best predictor for our variable, in any combination.
There are risks to this method: our sample size is smaller than I’d like, and this may lead to over-fit of outlier MSOAs.
It does mean we don’t need to worry about transformations or correlations: we can return to our original dataset, and let the model identify the strongest predictors.
# we remove rows where our main error is na or missing
rf_msoa_matrix <- drop_na(msoa_matrix_numeric, RPDRobberyShifted)
clean_rf_matrix <- rf_msoa_matrix[ , colSums(is.na(rf_msoa_matrix)) == 0]
#then we drop out any values that directly predict our error.
drop<- c("burglaryActual","burglaryError","burglaryPercentError","burglaryPredicted","robberyActual","robberyPredicted","robberyError","robberyPercentError", "RPDBurglaryShifted", "RPDRobberyShifted")
#remove out selected columns
data<- clean_rf_matrix[,!(names(clean_rf_matrix) %in% drop)]
names(clean_rf_matrix)<-make.names(names(clean_rf_matrix),unique = TRUE)
drop<- c("burglaryActual","burglaryError","robberyActual","robberyError","RPDBurglary","RPDRobbery","robberyActualShifted","robberyPredictedShifted","burglaryActualShifted","burglaryPredictedShifted", "burglaryPredicted", "burglaryPercentError", "robberyPredicted", "robberyPercentError")
#drop<- c("burglaryPercentError","burglaryPredicted","robberyPredicted","robberyPercentError")
We start with modelling our relative rate of burglary shift. We divide our sample into a training set which we’ll use to train the model, and a test set we’ll use to verify accuracy and validity.
sample = sample.split(data$RPDburglaryShifted, SplitRatio = 0.7)
train = subset(data, sample == TRUE)
test = subset(data, sample == FALSE)
RPDburglaryShifted ~ .,
data=train,
summary(rf_burglary) Length Class Mode
call 1 -none- -none- numeric
mse
rsq 500 -none- numeric
oob.times 685 -none- numeric -none- numeric
localImportance 0 -none- NULL
ntree 1 numeric
forest 11 -none- list
coefs 0 -none- NULL
y 685 0 -none- NULL
terms 3 prediction <-predict(rf_burglary, test)
Metrics::rmse(test$RPDburglaryShifted, prediction)[1] 0.2086745While machine learning models were once considered opaque and difficult to interpret, several libraries now offer functionality to explain predictions. Here I use DALEX to do just this.
rf_explainer_burglary <- DALEX::explain(rf_burglary, data=train, y= train$RPDburglaryShifted)Preparation of a new explainer is initiated
-> model label : ( default )
-> data : 685 rows 211 cols
-> data : tibble converted into a data.frame
-> target variable : -> predict function : yhat.randomForest will be used ( default )
-> predicted values : No value for predict function target column. ( default )
randomForest , ver. 4.6.14 , taskdefault )
-> predicted values : numerical, min = -0.5780085 , mean = -0.08106178 , max = 0.3933345
-> residual function : difference between y and yhat ( default )
-> residuals : numerical, min = -0.2656502 , mean = -0.000773038 , max = 0.3310273
A new explainer has been created! rf_perf_burg
rmse : 0.07978734
r2 : 0.8458048
mad : 0.04547341
Residuals:
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
-0.265650178 -0.088750799 -0.057666182 -0.039718463 -0.023652909 -0.007873967 0.008104222 0.029419548 0.052525556 0.089675687 0.331027295 We can see that our model significantly outperforms our best linear models: the R2 suggests that almost 85% of the variance is correctly interpreted, and our rmse (root mean squared error) is around 0.2 on our test set.
This model would be ill-suited to prediction or operationalising -it is a default forecast with no hyper-parameter tuning, and no consideration of error rates - but using tools like DALEX, we can identify which predictors the model identifies as most important.
model_parts_burg <-model_parts(rf_explainer_burglary)
model_parts_burg variable mean_dropout_loss label
1 RPDburglaryShifted 0.07978734 randomForest
3 Road.Casualties.2010.Fatal 0.07989223 randomForest
4 Road.Casualties.2011.Fatal 0.07990359 randomForest
5 Road.Casualties.2012.Fatal 0.07994289 randomForest
6 Age.Structure..2011.Census..All.Ages 0.08016856 randomForest
7 Mid.year.Estimate.totals.All.Ages.2010 0.08020098 randomForest
8 Car.or.van.availability..2011.Census..4.or.more.cars.or.vans.in.household.... 0.08023165 randomForest
9 UsualRes 0.08027333 randomForest
10 randomForest
12 Households..2011..All.Households 0.08034082 randomForest
13 Mid.year.Estimate.totals.All.Ages.2011 0.08034805 randomForest
14 Ethnic.Group..2011.Census..White.... 0.08035199 randomForest
15 Mid.year.Estimate.totals.All.Ages.2012 0.08035832 randomForest
16 HholdRes 0.08036393 randomForest
17 Qualifications..2011.Census..Highest.level.of.qualification..Other.qualifications 0.08036488 randomForest
18 randomForest
20 Car.or.van.availability..2011.Census..2.cars.or.vans.in.household
22 Country.of.Birth..2011..Not.United.Kingdom.... 0.08044878 randomForest Mid.year.Estimate.totals.All.Ages.2006 0.08047061 randomForest
25 Tenure..2011..Owned..Owned.outright.... 0.08047637 randomForest
26 Ethnic.Group..2011.Census..BAME.... 0.08047866 randomForest
27 Ethnic.Group..2011.Census..BAME 0.08049934 randomForest
28 Religion..2011..Other.religion.... 0.08050526 randomForest
29 Dwelling.type..2011..Household.spaces.with.at.least.one.usual.resident 0.08053065 randomForest
30 Hholds 0.08054706 randomForest
31 Household.Language..2011..At.least.one.person.aged.16.and.over.in.household.has.English.as.a.main.language 0.08057152 randomForest
32 Health..2011.Census..Day.to.day.activities.limited.a.little 0.08059003 randomForest
33 Religion..2011..Sikh.... 0.08059177 randomForest
34 Population.Density.Persons.per.hectare..2012. 0.08059472 randomForest
35 Household.Language..2011....of.households.where.no.people.in.household.have.English.as.a.main.language 0.08060243 randomForest
36 Road.Casualties.2011.Serious 0.08060476 randomForest
37 Household.Language..2011..No.people.in.household.have.English.as.a.main.language 0.08060958 randomForest
38 Car.or.van.availability..2011.Census..Sum.of.all.cars.or.vans.in.the.area 0.08064046 randomForest
39 Country.of.Birth..2011..Not.United.Kingdom 0.08066560 randomForest
42 Age.Structure..2011.Census..30.44 0.08066940 randomForest
43 Qualifications..2011.Census..Highest.level.of.qualification..Level.1.qualifications 0.08067143 randomForest
44 Car.or.van.availability..2011.Census..3.cars.or.vans.in.household.... 0.08067284 randomForest
45 Tenure..2011..Owned..Owned.with.a.mortgage.or.loan 0.08067322 randomForest
46 Country.of.Birth..2011..United.Kingdom.... 0.08068890 randomForest
47 Economic.Activity..2011.Census..Economically.active..Total 0.08069008 randomForest
48 Mid.year.Estimate.totals.All.Ages.2004 0.08069682 randomForest
49 Country.of.Birth..2011..United.Kingdom 0.08071084 randomForest
50 Mid.year.Estimate.totals.All.Ages.2003 0.08071974 randomForest
51 Lone.Parents..2011.Census..All.lone.parent.housholds.with.dependent.children 0.08073043 randomForest
52 Health..2011.Census..Day.to.day.activities.limited.a.little.... 0.08075471 randomForest
55 Tenure..2011..Social.rented.... 0.08076387 randomForest
56 PopDen 0.08076430 randomForest
57 Mid.year.Estimates.2012..by.age.60.64 0.08077331 randomForest
58 Mid.year.Estimate.totals.All.Ages.2008 0.08077994 randomForest
59 Religion..2011..No.religion.... 0.08078100 randomForest
60 House.Prices.Sales.2010 0.08078206 randomForest
61 Mid.year.Estimate.totals.All.Ages.2009 0.08078457 randomForest
62 Economic.Activity..2011.Census..Economically.inactive.. 0.08079916 randomForest
63 Ethnic.Group..2011.Census..Asian.Asian.British 0.08080477 randomForest
64 Car.or.van.availability..2011.Census..3.cars.or.vans.in.household 0.08080990 randomForest
65 Mid.year.Estimates.2012..by.age.85.89 0.08081135 randomForest
66 Tenure..2011..Owned..Owned.with.a.mortgage.or.loan.... 0.08082316 randomForest
67 Age.Structure..2011.Census..16.29 0.08083177 randomForest
68 Qualifications..2011.Census..Highest.level.of.qualification..Level.4.qualifications.and.above 0.08083667 randomForest
69 Age.Structure..2011.Census..45.64 0.08083957 randomForest
70 Mid.year.Estimates.2012..by.age.55.59 0.08084109 randomForest
71 Economic.Activity..2011.Census..Economically.active.. 0.08084118 randomForest
72 Religion..2011..Buddhist.... 0.08084915 randomForest
73 House.Prices.Median.House.Price.....2011 0.08085332 randomForest
74 Religion..2011..Hindu.... 0.08085661 randomForest
75 Central.Heating..2011.Census..Households.with.central.heating.... 0.08085781 randomForest
76 Qualifications..2011.Census..Highest.level.of.qualification..Level.2.qualifications 0.08086017 randomForest
77 Ethnic.Group..2011.Census..Other.ethnic.group 0.08086059 randomForest
78 Economic.Activity..2011.Census..Economically.active..Unemployed 0.08086791 randomForest
79 Religion..2011..Jewish.... 0.08087423 randomForest
80 Dwelling.type..2011..Whole.house.or.bungalow..Semi.detached.... 0.08087529 randomForest
81 Household.Composition..2011..Numbers.One.person.household 0.08087823 randomForest
82 Mid.year.Estimates.2012..by.age.80.84 0.08088729 randomForest
83 Household.Composition..2011..Numbers.Lone.parent.household 0.08088891 randomForest 0.08092207
87 Household.Composition..2011..Numbers.Couple.household.without.dependent.children 0.08093548 randomForest
88 Adults.in.Employment..2011.Census..No.adults.in.employment.in.household..With.dependent.children 0.08093596 randomForest
89 House.Prices.Median.House.Price.....2005 0.08094486 randomForest
90 Mid.year.Estimate.totals.All.Ages.2002 0.08094711 randomForest
91 Income.Deprivation..2010....living.in.income.deprived.households.reliant.on.means.tested.benefit 0.08094734 randomForest
92 Economic.Activity..2011.Census..Economically.inactive..Total 0.08095970 randomForest
93 Dwelling.type..2011..Flat..maisonette.or.apartment 0.08097084 randomForest
94 Adults.in.Employment..2011.Census....of.households.with.no.adults.in.employment..With.dependent.children 0.08097164 randomForest
95 Mid.year.Estimates.2012..by.age.15.19 0.08097487 randomForest
96 Qualifications..2011.Census..Highest.level.of.qualification..Apprenticeship 0.08099627 randomForest
97 Road.Casualties.2010.Serious 0.08099896 randomForest
98
100 Car.or.van.availability..2011.Census..No.cars.or.vans.in.household 0.08100307 randomForest
101 Qualifications..2011.Census..Schoolchildren.and.full.time.students..Age.18.and.over 0.08102459 randomForest
102 House.Prices.Median.House.Price.....2006 0.08102808 randomForest
103 Car.or.van.availability..2011.Census..1.car.or.van.in.household.... 0.08102910 randomForest
104 House.Prices.Sales.2011...130 0.08103290 randomForest
105 House.Prices.Median.House.Price.....2010 0.08103351 randomForest
106 Religion..2011..No.religion 0.08104129 randomForest
107 Religion..2011..Muslim.... 0.08104171 randomForest
108 randomForest
110 Dwelling.type..2011..Whole.house.or.bungalow..Semi.detached 0.08107212 randomForest
111 Tenure..2011..Private.rented 0.08107812 randomForest
112 Household.Composition..2011..Numbers.Couple.household.with.dependent.children randomForest
114 Dwelling.type..2011..Flat..maisonette.or.apartment.... 0.08110023 randomForest
117 Mid.year.Estimates.2012..by.age.90. 0.08110115 randomForest
118 House.Prices.Median.House.Price.....2009 0.08112692 randomForest
119 House.Prices.Sales.2013.p. 0.08112842 randomForest
120 Tenure..2011..Social.rented 0.08113023 randomForest
121 Mid.year.Estimates.2012..by.age.5.9 0.08113618 randomForest
122 House.Prices.Median.House.Price.....2008 0.08114446 randomForest
123 Household.Composition..2011..Numbers.Other.household.Types 0.08116352 randomForest
124 Obesity.Percentage.of.the.population.aged.16..with.a.BMI.of.30...modelled.estimate..2006.2008 0.08116727 randomForest
125 Car.or.van.availability..2011.Census..1.car.or.van.in.household 0.08116794 randomForest
126 Health..2011.Census..Very.good.health.... 0.08118887 randomForest
127 Mid.year.Estimates.2012..by.age.25.29 0.08120511 randomForest
128 Religion..2011..Hindu 0.08122569 randomForest
129 Health..2011.Census..Bad.health 0.08123266 randomForest
130 Mid.year.Estimates.2012..by.age...65. 0.08123475 randomForest
131 Land.Area.Hectares 0.08124618 randomForest
132 Ethnic.Group..2011.Census..Asian.Asian.British.... 0.08125689 randomForest
133 House.Prices.Sales.2009 0.08127707 randomForest
134 Qualifications..2011.Census..No.qualifications 0.08128507 randomForest
135 Mid.year.Estimates.2012..by.age.30.34 0.08131069 randomForest
136 Ethnic.Group..2011.Census..Black.African.Caribbean.Black.British.... 0.08131640 randomForest
137 Age.Structure..2011.Census..65. 0.08131793 randomForest
138 Mid.year.Estimates.2012..by.age.75.79 0.08132623 randomForest
139 Ethnic.Group..2011.Census..Black.African.Caribbean.Black.British 0.08132628 randomForest
140 Ethnic.Group..2011.Census..White 0.08133305 randomForest
141 Mid.year.Estimates.2012..by.age.50.54 0.08134709 randomForest 0.08136569 0.08136868 randomForest
146 randomForest
148 House.Prices.Median.House.Price.....2012 0.08141885 randomForest
149 Health..2011.Census..Fair.health.... 0.08142456 randomForest
150 AvHholdSz 0.08142887 randomForest
151 Life.Expectancy.Males 0.08143464 randomForest
152 Religion..2011..Religion.not.stated.... 0.08144721 randomForest
153 Religion..2011..Buddhist 0.08146208 randomForest
154 Low.Birth.Weight.Births..2007.2011..LCL...Lower.confidence.limit 0.08146848 randomForest
155 Religion..2011..Sikh 0.08149141 randomForest
156 Health..2011.Census..Very.good.health 0.08149203 randomForest
157 Low.Birth.Weight.Births..2007.2011..Low.Birth.Weight.Births.... 0.08149302 randomForest
158 Mid.year.Estimates.2012..by.age.35.39 0.08149439 randomForest
159 Health..2011.Census..Day.to.day.activities.limited.a.lot 0.08149561 randomForest
160 Tenure..2011..Owned..Owned.outright 0.08149998 randomForest
161 Road.Casualties.2010.Slight 0.08150819 randomForest
162 Low.Birth.Weight.Births..2007.2011..UCL...Upper.confidence.limit 0.08151073 randomForest
163 Road.Casualties.2012.Serious 0.08153523 randomForest
164 Tenure..2011..Private.rented.... 0.08154695 randomForest
165 House.Prices.Sales.2007 0.08156845 randomForest
166 Health..2011.Census..Day.to.day.activities.not.limited.... 0.08164471 randomForest
167 Road.Casualties.2012.2012.Total 0.08166058 randomForest
168 Road.Casualties.2010.2010.Total 0.08167894 randomForest
169 Religion..2011..Other.religion 0.08168736 randomForest
170 Road.Casualties.2012.Slight 0.08169606 randomForest
171 Mid.year.Estimates.2012..by.age.40.44 0.08173699 randomForest
172 Mid.year.Estimate.totals.All.Ages.2007 0.08173808 randomForest
173 Health..2011.Census..Day.to.day.activities.limited.a.lot.... 0.08174045 randomForest
174 Ethnic.Group..2011.Census..Mixed.multiple.ethnic.groups randomForest
177 Qualifications..2011.Census..Highest.level.of.qualification..Level.3.qualifications 0.08175634 randomForest
178 Dwelling.type..2011..Whole.house.or.bungalow..Detached 0.08177575 randomForest
179 Religion..2011..Christian 0.08179686 randomForest
180 House.Prices.Sales.2005 0.08182753 randomForest
181 House.Prices.Median.House.Price.....2007 0.08184118 randomForest
182 Health..2011.Census..Bad.health.... 0.08185248 randomForest
183 Health..2011.Census..Good.health.... 0.08194459 randomForest
184 total_robberies 0.08196227 randomForest
185 House.Prices.Sales.2006 0.08196583 randomForest
186 Dwelling.type..2011..Household.spaces.with.no.usual.residents.... 0.08200226 randomForest
187 Road.Casualties.2011.2011.Total 0.08203031 randomForest
188 Dwelling.type..2011..Whole.house.or.bungalow..Terraced..including.end.terrace. 0.08204300 randomForest
189 Mid.year.Estimates.2012..by.age.20.24 0.08210720 randomForest
190 Mid.year.Estimates.2012..by.age.65.69 0.08211403 randomForest
191 Dwelling.type..2011..Whole.house.or.bungalow..Detached.... 0.08213368 randomForest
192 Household.Income.Estimates..2011.12..Total.Mean.Annual.Household.Income.... 0.08215211 randomForest
193 Mid.year.Estimates.2012..by.age.70.74 0.08221997 randomForest
194 Incidence.of.Cancer.Breast.Cancer 0.08229737 randomForest
195 Road.Casualties.2011.Slight 0.08231561 randomForest
196 Economic.Activity..2011.Census..Unemployment.Rate 0.08239690 randomForest
197 Lone.Parents..2011.Census..Lone.parent.not.in.employment.. 0.08246826 randomForest
198 0.08262825 randomForest
200 Dwelling.type..2011..Household.spaces.with.no.usual.residents 0.08264973 randomForest
201 Life.Expectancy.Females 0.08276533 randomForest
202 Dwelling.type..2011..Whole.house.or.bungalow..Terraced..including.end.terrace..... 0.08284255 randomForest
203 Religion..2011..Jewish 0.08293921 randomForest
204 Incidence.of.Cancer.All 0.08306588 randomForest
205 RPDRobberyShifted 0.08370759 randomForest
206 Household.Composition..2011..Percentages.Couple.household.with.dependent.children 0.08393367 randomForest
207 House.Prices.Sales.2008 0.08433984 randomForest
208 Household.Composition..2011..Percentages.Couple.household.without.dependent.children 0.08459338 randomForest
209 Mid.year.Estimates.2012..by.age...0.to.14 0.08578863 randomForest
210 total_burglaries 0.08833999 randomForest
211 ComEstRes 0.09121371 randomForest
212 Mid.year.Estimates.2012..by.age.0.4 0.09159861 randomForest
213 _baseline_ 0.24106307 randomForestplot(model_parts_burg, max_vars=25)
The three most important features our model highlights are: - the age composition of the population - the number of residents which are in commercial property - the historic number of burglaries
This seems to corroborate our previous regression model. To further unpick these trends, we can use Partial Dependence Plots to identify how the model prediction shifts with these values.
pdp <- model_profile(rf_explainer_burglary)
plot(pdp, variables="total_burglaries")
We still see a strong association between a high number of historic, and a large reduction during the lockdown period
plot(pdp, variables= "Mid.year.Estimates.2012..by.age.0.4")
MSOAs with very large very young populations seem to have a less strong burglary lockdown decrease (though this is likely heavily associated to deprivation)
plot(pdp, variables="ComEstRes")
hist(msoa_matrix_numeric$ComEstRes)
By combining the distribution of commercial residents by MSOA with our PDP, we can see that those MSOAs that are most densely populated by commercial residents see the smallest “covid decrease”(suggesting that those MSOAs that are very heavily residential saw the sharpest drops).
plot(pdp, variables="House.Prices.Sales.2008")
Finally, we can see that those areas that experienced the highest volume of house sales experienced the lowest relative decrease in burglary.
This highlights the importance of identifying correlates in RF models - especially in a dataset where features are highly interlinked, association does not imply causation, and we should be wary of over-interpreting.
We can now repeat our process for our robbery shift.
sample = sample.split(data$RPDRobberyShifted, SplitRatio = 0.75)
train = subset(data, sample == TRUE)
test = subset(data, sample == FALSE)
rf_robbery <- randomForest(
RPDRobberyShifted ~ .,
data=train,
importance=TRUE
)
summary(rf_robbery) Length Class Mode
call -none- character
predicted 734 -none- numeric
rsq 500 -none- numeric
importance 420 -none- numeric
localImportance 0 0 -none- NULL
ntree 1 -none- numeric
mtry 1 -none- numeric
forest 11 -none- list
coefs 0 -none- NULL
y 734 -none- numeric
test 0 -none- NULL
inbag 0 -none- NULL
terms 3 terms call We’ve now trained a model. Let’s now use the DALEX library to understand it, and see how it performs.
rf_explainer_robbery <- DALEX::explain(rf_robbery, data=train, y= train$RPDRobberyShifted)Preparation of a new explainer is initiated
-> model label : randomForest ( default )
-> data : 734 rows 211 cols
-> data : tibble converted into a data.frame
-> target variable : 734 values
yhat.randomForest will be used ( default )
-> predicted values : No value for predict function target column. ( default )
4.6.14 , task regression ( )
-> predicted values : numerical, min = -0.9645244 , mean = -0.0470117 , max = 0.3797893
-> residual function : difference between y and yhat ( default )
-> residuals : numerical, min = -0.4903862 , mean = -0.001156124 , max = 0.2960573
rf_perf_rob <- model_performance(rf_explainer_robbery)
rf_perf_robMeasures for:
r2 : 0.8666274
mad : 0.02860048
Residuals:
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
-0.490386240 -0.062713767 -0.038399387 -0.024521257 -0.013158676 -0.004124880 0.004984989 0.019451443 0.035391571 0.063885825 0.296057263 model_parts_rob <-model_parts(rf_explainer_robbery)
model_parts_rob variable mean_dropout_loss label
1 _full_model_ 0.05952187 randomForest
2 RPDRobberyShifted 0.05952187 randomForest
3 Road.Casualties.2012.Fatal 0.05956864 randomForest
4 Road.Casualties.2010.Fatal 0.05958931 randomForest
5 Road.Casualties.2011.Fatal 0.05959538 randomForest
6 HholdRes 0.05970861 randomForest
7 Mid.year.Estimate.totals.All.Ages.2010 0.05973753 randomForest
8 Car.or.van.availability..2011.Census..4.or.more.cars.or.vans.in.household.... 0.05975769 randomForest
9 Mid.year.Estimate.totals.All.Ages.2011 0.05976137 randomForest
10 Health..2011.Census..Day.to.day.activities.not.limited 0.05977415 randomForest
11 AvHholdSz 0.05977911 randomForest
12 Age.Structure..2011.Census..All.Ages 0.05978227 randomForest
13 UsualRes 0.05979675 randomForest
14 Lone.Parents..2011.Census..All.lone.parent.housholds.with.dependent.children 0.05980643 randomForest
15 Mid.year.Estimate.totals.All.Ages.2006 0.05981345 randomForest
16 Car.or.van.availability..2011.Census..2.cars.or.vans.in.household 0.05986355 randomForest
17 Dwelling.type..2011..Household.spaces.with.at.least.one.usual.resident 0.05986499 randomForest
18 Tenure..2011..Owned..Owned.outright 0.05986769 randomForest 0.05988390 randomForest 0.05988822 randomForest
23 Road.Casualties.2010.Serious 0.05988899 randomForest
24 Adults.in.Employment..2011.Census....of.households.with.no.adults.in.employment..With.dependent.children 0.05988994 randomForest
25 House.Prices.Median.House.Price.....2009 0.05989951 randomForest
26 Economic.Activity..2011.Census..Economically.active..Total 0.05990367 randomForest
27 Religion..2011..Jewish.... 0.05991359 randomForest
28 Household.Language..2011..At.least.one.person.aged.16.and.over.in.household.has.English.as.a.main.language 0.05991422 randomForest
29 Dwelling.type..2011..Whole.house.or.bungalow..Detached.... 0.05991695 randomForest
30 Dwelling.type..2011..Whole.house.or.bungalow..Detached 0.05991703 randomForest
31 Dwelling.type..2011..Whole.house.or.bungalow..Semi.detached.... 0.05992179 randomForest
32 House.Prices.Median.House.Price.....2010 0.05992630 randomForest
33 Mid.year.Estimates.2012..by.age.70.74 0.05993161 randomForest
34 Car.or.van.availability..2011.Census..4.or.more.cars.or.vans.in.household 0.05993504 randomForest
35 randomForest
37 Mid.year.Estimate.totals.All.Ages.2007 0.05995053 randomForest
38 Mid.year.Estimate.totals.All.Ages.2004 0.05996353 randomForest
39 Mid.year.Estimate.totals.All.Ages.2012 0.05996873 randomForest
40 Dwelling.type..2011..Whole.house.or.bungalow..Semi.detached 0.05997034 randomForest
41 Age.Structure..2011.Census..65. 0.05997184 randomForest
42 Qualifications..2011.Census..Highest.level.of.qualification..Level.2.qualifications 0.05997500 randomForest
43 Religion..2011..Other.religion.... 0.05997794 randomForest
44 Health..2011.Census..Very.good.health 0.05997835 randomForest
45 House.Prices.Median.House.Price.....2005 0.05998924 randomForest
46 Tenure..2011..Social.rented.... 0.05999127 randomForest
47 Age.Structure..2011.Census..Working.age 0.05999621 randomForest
48 Mid.year.Estimates.2012..by.age...65. 0.06001570 randomForest
49 Car.or.van.availability..2011.Census..3.cars.or.vans.in.household
52 House.Prices.Median.House.Price.....2011 0.06004060 randomForest
53 Household.Composition..2011..Numbers.Lone.parent.household 0.06004582 randomForest
54 Obesity.Percentage.of.the.population.aged.16..with.a.BMI.of.30...modelled.estimate..2006.2008 0.06004786 randomForest
55 Car.or.van.availability..2011.Census..Sum.of.all.cars.or.vans.in.the.area 0.06004791 randomForest
56 Mid.year.Estimates.2012..by.age.80.84 0.06004946 randomForest
57 Mid.year.Estimates.2012..by.age.50.54 0.06005044 randomForest 0.06005591 randomForest
60 Age.Structure..2011.Census..30.44 0.06005931 randomForest
61 Car.or.van.availability..2011.Census..1.car.or.van.in.household 0.06006814 randomForest
62 Adults.in.Employment..2011.Census..No.adults.in.employment.in.household..With.dependent.children 0.06007092 randomForest
63 Households..2011..All.Households 0.06007556 randomForest
64 House.Prices.Median.House.Price.....2007 0.06007826 randomForest
65 Economic.Activity..2011.Census..Economically.inactive.. 0.06008955 randomForest
66 Household.Income.Estimates..2011.12..Total.Mean.Annual.Household.Income.... 0.06009647 randomForest randomForest
69 Qualifications..2011.Census..No.qualifications 0.06010873 randomForest
70 Lone.Parents..2011.Census..Lone.parents.not.in.employment 0.06011036 randomForest
71 Age.Structure..2011.Census..0.15 0.06011788 randomForest
72 Household.Composition..2011..Percentages.Lone.parent.household 0.06011867 randomForest
73 Mid.year.Estimates.2012..by.age.40.44 0.06012597 randomForest
74 House.Prices.Sales.2011...129 0.06012701 randomForest
75 Income.Deprivation..2010....living.in.income.deprived.households.reliant.on.means.tested.benefit 0.06012765 randomForest
76 Health..2011.Census..Bad.health.... 0.06013151 randomForest
77 House.Prices.Median.House.Price.....2013..p. 0.06013614 0.06016171 randomForest
80 Country.of.Birth..2011..United.Kingdom 0.06016239 randomForest
81 House.Prices.Sales.2013.p. 0.06016255 randomForest 0.06017141 randomForest
84 Economic.Activity..2011.Census..Unemployment.Rate 0.06017764 randomForest
85 Household.Composition..2011..Percentages.Couple.household.without.dependent.children 0.06019165 randomForest
86 House.Prices.Sales.2010 0.06019170 randomForest
87 House.Prices.Sales.2005 0.06019370 randomForest
88 House.Prices.Sales.2006 0.06019433 randomForest
89 Mid.year.Estimates.2012..by.age.55.59 0.06019794 randomForest
90 Health..2011.Census..Day.to.day.activities.limited.a.little.... 0.06019800 randomForest
91 Economic.Activity..2011.Census..Economically.active.. 0.06020157 randomForest
92 Tenure..2011..Owned..Owned.outright.... 0.06020697 randomForest
93 Mid.year.Estimates.2012..by.age.30.34 0.06020928 randomForest
94 House.Prices.Median.House.Price.....2012 0.06020958 randomForest
95 Mid.year.Estimates.2012..by.age.45.49 0.06022425 randomForest
96 Central.Heating..2011.Census..Households.with.central.heating.... 0.06023901 randomForest
97 Religion..2011..Jewish 0.06024087 randomForest
98 Health..2011.Census..Very.bad.health.... 0.06024264 randomForest
99 Household.Composition..2011..Numbers.Couple.household.with.dependent.children 0.06025090 randomForest
100 Mid.year.Estimates.2012..by.age.75.79 0.06025885 randomForest
101 Mid.year.Estimate.totals.All.Ages.2002 0.06027020 randomForest
102 Household.Income.Estimates..2011.12..Total.Median.Annual.Household.Income.... 0.06027310 randomForest
103 Household.Composition..2011..Numbers.Couple.household.without.dependent.children 0.06027733 randomForest
104 Qualifications..2011.Census..Highest.level.of.qualification..Level.1.qualifications 0.06027961 randomForest
105 Dwelling.type..2011..Household.spaces.with.at.least.one.usual.resident.... 0.06029249 randomForest
106 Tenure..2011..Owned..Owned.with.a.mortgage.or.loan.... 0.06029446 randomForest
107 Dwelling.type..2011..Household.spaces.with.no.usual.residents 0.06031695 randomForest
108 Religion..2011..No.religion 0.06031908 randomForest
109 randomForest 0.06032636 randomForest
113 Health..2011.Census..Day.to.day.activities.limited.a.lot.... 0.06032787 randomForest
114 Mid.year.Estimates.2012..by.age.5.9 0.06032920 randomForest
115 Road.Casualties.2012.Serious 0.06033966 randomForest
116 Health..2011.Census..Good.health 0.06034929 randomForest
117 Religion..2011..No.religion.... 0.06036163 randomForest
118 Income.Deprivation..2010....of.people.aged.over.60.who.live.in.pension.credit.households 0.06036686 randomForest
119 Religion..2011..Hindu.... 0.06036907 randomForest
120 Dwelling.type..2011..Flat..maisonette.or.apartment 0.06037343 randomForest
121 Health..2011.Census..Fair.health.... 0.06037424 randomForest
122 Dwelling.type..2011..Whole.house.or.bungalow..Terraced..including.end.terrace. 0.06037583 randomForest
123 Lone.Parents..2011.Census..Lone.parent.not.in.employment.. 0.06037930 0.06038862 randomForest
126 Economic.Activity..2011.Census..Economically.active..Unemployed 0.06039167 randomForest
127 House.Prices.Median.House.Price.....2008 0.06039358 randomForest
128 Religion..2011..Christian.... 0.06039596 randomForest
129 Mid.year.Estimates.2012..by.age.20.24 0.06039727 randomForest
130 Religion..2011..Other.religion 0.06039886 randomForest
131 ComEstRes 0.06040057 randomForest
132 Household.Composition..2011..Percentages.Other.household.Types 0.06040106 randomForest
133 House.Prices.Sales.2007
135 Household.Composition..2011..Numbers.Other.household.Types 0.06040495 randomForest
138 Dwelling.type..2011..Household.spaces.with.no.usual.residents.... 0.06044575 randomForest
139 Religion..2011..Muslim.... 0.06045004 randomForest
140 Health..2011.Census..Day.to.day.activities.limited.a.little 0.06045210 randomForest
141 Tenure..2011..Private.rented.... 0.06045264 randomForest
142 Ethnic.Group..2011.Census..White 0.06045855 Low.Birth.Weight.Births..2007.2011..LCL...Lower.confidence.limit 0.06046500 randomForest
145 Dwelling.type..2011..Flat..maisonette.or.apartment.... 0.06046511 randomForest
146 House.Prices.Sales.2008 0.06046573 randomForest
147 Religion..2011..Christian 0.06046620 randomForest
148 Car.or.van.availability..2011.Census..2.cars.or.vans.in.household.... 0.06046913 randomForest
149 Ethnic.Group..2011.Census..Asian.Asian.British 0.06048039 randomForest
150 Religion..2011..Hindu 0.06049182 randomForest
151 Household.Composition..2011..Percentages.One.person.household 0.06049665 randomForest
152 House.Prices.Sales.2009 0.06049784 randomForest
153 House.Prices.Sales.2011...130 0.06049885 Health..2011.Census..Very.good.health.... 0.06051842 randomForest
156 Low.Birth.Weight.Births..2007.2011..Low.Birth.Weight.Births.... 0.06052278 randomForest
157 Health..2011.Census..Very.bad.health 0.06052527 randomForest
158 Health..2011.Census..Fair.health 0.06052537 randomForest
159 randomForest
161 Ethnic.Group..2011.Census..Asian.Asian.British.... 0.06054116 randomForest
162 Low.Birth.Weight.Births..2007.2011..UCL...Upper.confidence.limit 0.06055614 randomForest
163 Ethnic.Group..2011.Census..Black.African.Caribbean.Black.British 0.06056211 randomForest
164 Qualifications..2011.Census..Highest.level.of.qualification..Other.qualifications 0.06056219 randomForest
165 Mid.year.Estimates.2012..by.age.0.4 0.06056244 randomForest
166 Population.Density.Persons.per.hectare..2012. 0.06057661 randomForest
167 Religion..2011..Religion.not.stated.... 0.06058630 randomForest
168 Religion..2011..Religion.not.stated 0.06058679 randomForest
169 Religion..2011..Sikh 0.06059155 randomForest
170 Incidence.of.Cancer.Breast.Cancer 0.06062165 randomForest
171 Country.of.Birth..2011..United.Kingdom.... 0.06062725 randomForest
172 Ethnic.Group..2011.Census..BAME 0.06066563 randomForest
173 Country.of.Birth..2011..Not.United.Kingdom 0.06068361 randomForest
174 Mid.year.Estimates.2012..by.age...0.to.14 0.06068807 randomForest
175 Tenure..2011..Owned..Owned.with.a.mortgage.or.loan 0.06068995 randomForest
176 Ethnic.Group..2011.Census..Mixed.multiple.ethnic.groups 0.06069163 randomForest
177 Qualifications..2011.Census..Highest.level.of.qualification..Level.3.qualifications 0.06072193 randomForest
178 Car.or.van.availability..2011.Census..1.car.or.van.in.household.... 0.06074983 randomForest
179 Mid.year.Estimates.2012..by.age.15.19 0.06075380 randomForest
180 Household.Language..2011....of.people.aged.16.and.over.in.household.have.English.as.a.main.language 0.06075892 randomForest
181 Health..2011.Census..Bad.health 0.06076332 randomForest
182 Health..2011.Census..Day.to.day.activities.limited.a.lot 0.06077496 randomForest
183 Ethnic.Group..2011.Census..Black.African.Caribbean.Black.British.... 0.06082769 randomForest
184 Household.Language..2011..No.people.in.household.have.English.as.a.main.language 0.06085115 randomForest
185 Car.or.van.availability..2011.Census..No.cars.or.vans.in.household 0.06085605 randomForest
186 Car.or.van.availability..2011.Census..No.cars.or.vans.in.household.... 0.06087480 randomForest
187 Health..2011.Census..Good.health.... 0.06088495 randomForest
188 Household.Language..2011....of.households.where.no.people.in.household.have.English.as.a.main.language 0.06090083 Ethnic.Group..2011.Census..White.... 0.06100824 randomForest
191 Life.Expectancy.Females 0.06113049 randomForest
192 Economic.Activity..2011.Census..Economically.inactive..Total 0.06114768 randomForest
193 Age.Structure..2011.Census..16.29 0.06116905 randomForest
194 Ethnic.Group..2011.Census..BAME.... 0.06120881 randomForest
195 Life.Expectancy.Males 0.06122637 randomForest
196
198 Incidence.of.Cancer.All 0.06157997 randomForest
199 RPDburglaryShifted
201 Road.Casualties.2012.2012.Total 0.06221397 randomForest
202 Religion..2011..Buddhist.... 0.06230384 randomForest 0.06238647 randomForest
205 Road.Casualties.2011.2011.Total 0.06269966 randomForest
206 Road.Casualties.2010.2010.Total 0.06275241 randomForest
207 Religion..2011..Buddhist 0.06309183 randomForest
208 Ethnic.Group..2011.Census..Other.ethnic.group 0.06343477 randomForest
209 Road.Casualties.2010.Slight 0.06363878 randomForest
210 Ethnic.Group..2011.Census..Other.ethnic.group.... 0.06459224 randomForest
211 total_burglaries 0.06882403 randomForest
212 total_robberies randomForestplot(model_parts_rob, max_vars=25)
pdp_rob <- model_profile(rf_explainer_robbery)plot(pdp_rob, variables="total_robberies")
plot(pdp_rob, variables="total_burglaries")
The effect of historic crime effects again appears like a reliable predictor of a robbery covid shift: those MSOAs with the highest number of burglaries and robberies see strong decreases in robbery (though the association with burglary is not clear cut, suggesting other interaction effects may be driving this)
plot(pdp_rob, variables="Road.Casualties.2011.2011.Total")
Road casualties is another strong relationship. This could be a proxy for deprivation, but I suggest this is more down to geographic features - road casualties are probably rarer in denser urban environments, most likely to be affected by lockdown.
plot(pdp_rob, variables="Ethnic.Group..2011.Census..Other.ethnic.group....")
hist(data$Ethnic.Group..2011.Census..Other.ethnic.group....)
Finally, we see a demographic predictor linked to “other ethnic group”. I’m not clear how to interpret this, but it suggests that those MSOAs that are most diverse (and have the largest representation by these ethnic groups) experienced the strongest decreases in robbery during lockdown.
Together, these analyses suggest that the crime drop for robbery and burglary during national lockdown was significantly affected by distinct local factors. For both offences, historical crime trends play a role, with high crime areas experiencing a relatively larger drop in both burglary and robbery.
Beyond that, the drivers vary for each offence type: burglary was driven by the composition of the residential population, with heavily residential areas, and areas with a relatively “stable” population as measured by low housing sales also saw larger decrease - this is likely due to the increased number of empty residential properties.
Robbery decreases conversely, are well associated with high numbers of road casualties, as well as the ethnic makeup of the local population - this is likely due to an association with denser, more urban areas, with lower street speeds, and a possible link to deprivation, whereby minority population were least able to work from home, and were likely to still present as available targets for robbery.
LS0tDQp0aXRsZTogIkxlYXJuaW5nIFIgLSBFeHBsb3JpbmcgdGhlIENPVklEIENyaW1lIEVmZmVjdCBpbiBMb25kb24iDQpvdXRwdXQ6IGh0bWxfbm90ZWJvb2sNCi0tLQ0KIyMgRXhwbG9yaW5nIFByZWRpY3RvcnMgb2YgQ09WSUQgQ3JpbWUgU2hpZnRzDQpUaGUgbG9ja2Rvd24gYW5kIHNvY2lhbCBkaXN0YW5jaW5nIG1lYXN1cmVzIHRoYXQgd2VyZSBicm91Z2h0IGluIHRocm91Z2hvdXQgdGhlIHdvcmxkIHRvIHRhY2tsZSBDT1ZJRCBpbiAyMDIwIGhhdmUgaGFkIGEgc2lnbmlmaWNhbnQsIHdpZGVzcHJlYWQgZWZmZWN0IG9uIGNyaW1lLiBJbiB0aGlzIG5vdGVib29rLCBJIHVzZSBwdWJsaWMgTG9uZG9uIGNyaW1lIGRhdGEgb24gcm9iYmVyeSBhbmQgYnVyZ2xhcnkgdG8gZXhhbWluZSB3aGVyZSB0aGlzICJDT1ZJRCBjcmltZSBzaGlmdCIgd2FzIHN0cm9uZ2VzdCwgYW5kIHdoZXRoZXIgYW55IHNwZWNpZmljIGRyaXZlcnMgb3IgY29ycmVsYXRlcyBjYW4gYmUgaWRlbnRpZmllZC4gIA0KDQpPdXIgZmluZGluZ3Mgc3VnZ2VzdCB0aGF0IHRoZSByZWxhdGl2ZSBjaGFuZ2UgaW4gYnVyZ2xhcnkgYW5kIHJvYmJlcnkgdHJlbmRzIGR1cmluZyAibG9ja2Rvd24iIGluIEFwcmlsIGFuZCBNYXkgMjAyMCB3YXMgaGVhdmlseSBhZmZlY3RlZCBieSBsb2NhbCBjaGFyYWN0ZXJpc3RpY3M6IGFyZWFzIHdpdGggYSBoaWdoIHJlc2lkZW50aWFsIHBvcHVsYXRpb24gc2F3IHRlaCBzaGFycGVzdCBkZWNyZWFzZXMgaW4gYnVyZ2xhcnkgKGxpa2VseSBkdWUgdG8gYSByZWR1Y3Rpb24gaW4gYXZhaWxhYmxlIHRhcmdldHMpIHdoaWxlIHRoZSByZWR1Y3Rpb24gaW4gcm9iYmVyaWVzIGluc3RlYWQgc2VlbSB0byBiZSBkcml2ZW4gYnkgZ2VvZ3JhcGhpYyBmZWF0dXJlcyBhbmQgcHJveHkgaW5kaWNhdG9ycyBvZiBkZXByaXZhdGlvbiAocG90ZW50aWFsbHkgc3VnZ2VzdGluZyBtb3JlIGF2YWlsYWJsZSB0YXJnZXRzIGZvciByb2JiZXJ5IGluIGNvbW11bml0aWVzIGxlYXN0IGFibGUgdG8gd29yayBmb3IgZnJvbSkuDQoNClRoZSBwcmltYXJ5IHB1cnBvc2Ugb2YgdGhpcyBleGVyY2lzZSB3YXMgdG8gbGVhcm4gUiAtIEkndmUgcHJldmlvdXNseSB3b3JrZWQgZW50aXJlbHkgaW4gUHl0aG9uLCB3aGljaCBpcyBtb3JlIHRoYW4gc3VmZmljaWVudCA5OSUgb2YgdGhlIHRpbWUsIGJ1dCBoYXMgYXQgdGltZXMgcHJvdmVkIGEgYmxvY2tlciB3aGVuIEkgd2FudCB0byB0YWNrbGUgc29tZSBtb3JlIGV4cGVyaW1lbnRhbCBnZW9zcGF0aWFsIG1ldGhvZHMgb3IgdG9vbHMgZ2VhcmVkIHRvd2FyZHMgdGhlIGFjYWRlbWljIGNvbW11bml0eS4gIFdpdGggdGhhdCBpbiBtaW5kLCB0aGlzIGlzIGxpa2VseSB0byBiZSBhIGxpdHRsZSBtZXNzeSwgYW5kIEknbGwgYWltIHRvIGNvbmRlbnNlIG15IG1haW4gbGVzc29ucyBpbnRvIGEgYmxvZyBwb3N0IGluIHRoZSBmdXR1cmUuIFRoZSBtb2RlbHMgYXJlIG5vdCBoZWF2aWx5IHR1bmVkIChhaW1pbmcgdG8gZXhwbG9yZSBjb3JyZWxhdGVzIHJhdGhlciB0aGFuIHByb3ZpZGUgYWNjdXJhdGUgcHJlZGljdGlvbnMpIGFuZCBjZXJ0YWluIHByZWRpY3RvcnMgYXJlIGxpa2VseSB0byBjb3JyZWxhdGUgd2l0aCBlYWNoIG90aGVyLCBhbmQgYXMgc3VjaCBsaWtlbHkgZG8gbm90IGltcGx5IGRpcmVjdCBjYXVzYXRpb25zLg0KDQojIyMjIFJlc291cmNlcyBJJ3ZlIHVzZWQNCi0gTWF0dCBBc2hieSBDcmltZSBNYXBwaW5nIGNvdXJzZTogaHR0cHM6Ly9naXRodWIuY29tL21wamFzaGJ5L2NyaW1lbWFwcGluZy8NCi0gU3BhdGlhbCBNb2RlbGxpbmcgZm9yIERhdGEgU2NpZW50aXN0czogaHR0cHM6Ly9nZHNsLXVsLmdpdGh1Yi5pby9zYW4vDQotIFIgZm9yIERhdGEgU2NpZW5jZTogaHR0cHM6Ly9yNGRzLmhhZC5jby5uei9pbmRleC5odG1sDQotIEdlb2NvbXB1dGF0aW9uIHdpdGggUjogaHR0cHM6Ly9nZW9jb21wci5yb2JpbmxvdmVsYWNlLm5ldC8NCg0KDQoNCiMjIyBUYXNrcw0KDQoxLiBJbmdlc3QgYnVyZ2xhcnkgYW5kIHJvYmJlcnkgZGF0YSBhbmQgYXNzaWduIHRvIE1TT0ENCjIuIFByZWRpY3QgdHJlbmQgYnkgTVNPQQ0KMy4gSWRlbnRpZnkgQ09WSUQgZWZmZWN0L2Vycm9yIGJ5IE1TT0ENCjQuIE1vZGVsDQoNCiMjIyBEYXRhDQpJbiB0aGlzIGV4ZXJjaXNlLCBJJ20gdXNpbmcgdGhyZWUgeWVhcnMgb2YgTWV0cm9wb2xpdGFuIFBvbGljZSBTZXJ2aWNlIGRhdGEgZnJvbSBkYXRhLnBvbGljZS51ayAtIEkndmUgZG93bmxvYWRlZCB0aGVzZSBtYW51YWxseSByYXRoZXIgdGhhbiB1c2luZyB0aGVpciBBUEkuDQoNCiMjIDEuIEluZ2VzdCBidXJnbGFyeSBhbmQgcm9iYmVyeSBkYXRhIGFuZCBhc3NpZ24gdG8gTVNPQQ0KVW5saWtlIFB5dGhvbiBoYXMgdGhlIGJlbmVmaXQgb2YgYmVpbmcgImZvY3VzZWQiIC0gaXQncyBwcmltYXJ5aWx5IGEgdG9vbCBmb3IgYWNhZGVtaWNzL3NjaWVudGlzdHMsIHJhdGhlciB0aGFuIGEgcHJvZ3JhbW1pbmcgbGFuZ3VhZ2UuIFRoYXQgbWVhbnMgdGhlIGVjby1zeXN0ZW0gaXMgcmVmcmVzaGluZ2x5IGhlbHBmdWw6IGF0IGZpcnN0IGdsYW5jZSwgdGhlcmUgaXMgb25lIHByaW1hcnkgSURFLCBvbmUgcGFja2FnZSBsaWJyYXJ5LCBhbmQgaXQgYWxsIHdvcmtzLiBVbmxpa2UgUHl0aG9uLCBtb3N0IGxpYnJhcmllcyBhcmUgaW1wb3J0ZWQgZ2xvYmFsbHksIGFuZCB0aGF0IHdvcmtzIG9rYXkuLi4qbW9zdGx5Kg0KDQpbTVNPQXMgYXJlIGdlb2dyYXBoaWNhbCB1bml0cyBzcGVjaWZpY2FsbHkgZGVzaWduZWQgZm9yIGFuYWx5c2lzLCBhbmQgdG8gYmUgY29tcGFyYWJsZTogdGhleSBhbGwgaGF2ZSBhbiBhdmVyYWdlIHBvcHVsYXRpb24gb2YganVzdCBvdmVyIDgsMDAwLl0oaHR0cHM6Ly93d3cub25zLmdvdi51ay9tZXRob2RvbG9neS9nZW9ncmFwaHkvdWtnZW9ncmFwaGllcy9jZW5zdXNnZW9ncmFwaHkjb3V0cHV0LWFyZWEtb2EpDQoNCmBgYHtyLCByZXN1bHRzPSdoaWRlJ30NCiMgRGF0YSBtYW5pcHVsYXRpb24sIHRyYW5zZm9ybWF0aW9uIGFuZCB2aXN1YWxpc2F0aW9uDQpsaWJyYXJ5KHRpZHl2ZXJzZSkNCiMgTmljZSB0YWJsZXMNCmxpYnJhcnkoa2FibGVFeHRyYSkNCiMgU2ltcGxlIGZlYXR1cmVzIChhIHN0YW5kYXJkaXNlZCB3YXkgdG8gZW5jb2RlIHZlY3RvciBkYXRhIGllLiBwb2ludHMsIGxpbmVzLCBwb2x5Z29ucykNCmxpYnJhcnkoc2YpIA0KIyBTcGF0aWFsIG9iamVjdHMgY29udmVyc2lvbg0KbGlicmFyeShzcCkgDQojIFRoZW1hdGljIG1hcHMNCmxpYnJhcnkodG1hcCkgDQojIENvbG91ciBwYWxldHRlcw0KbGlicmFyeShSQ29sb3JCcmV3ZXIpIA0KIyBNb3JlIGNvbG91ciBwYWxldHRlcw0KbGlicmFyeSh2aXJpZGlzKQ0KI2dncGxvdCBvcmdhbmlhc3Rpb24NCmxpYnJhcnkoZ2dwdWJyKQ0KDQoNCmxpYnJhcnkocmFzdGVyKSAgIyByYXN0ZXIgZGF0YQ0KbGlicmFyeShyZ2RhbCkgICMgaW5wdXQvb3V0cHV0LCBwcm9qZWN0aW9ucw0KbGlicmFyeShyZ2VvcykgICMgZ2VvbWV0cnkgb3BzDQpsaWJyYXJ5KHNwZGVwKSAgIyBzcGF0aWFsIGRlcGVuZGVuY2UNCg0KbGlicmFyeShsdWJyaWRhdGUpICNkYXRlIGFuZCB0aW1lDQoNCiNyYW5kb20gZm9yZXN0IGFuZCBtZXRyaWNzDQpsaWJyYXJ5KE1ldHJpY3MpDQpsaWJyYXJ5KGNhcmV0KQ0KbGlicmFyeShyYW5kb21Gb3Jlc3QpDQpyZXF1aXJlKGNhVG9vbHMpDQpsaWJyYXJ5KERBTEVYKQ0KbGlicmFyeShkcGx5cikNCg0KDQoNCiNjb3JyZWxhdGlvbiBtYXRyaXgNCmxpYnJhcnkoY29ycnIpDQoNCg0KDQpgYGANCg0KSW1wb3J0aW5nIGRhdGEgZnJvbSBDU1YgZmlsZXMgYmVoYXZlcyBxdWl0ZSBzaW1pbGFybHkgdG8gUHl0aG9uLiBUbyBidWlsZCBvdXIgcHJvY2Vzcywgd2UnbGwgc3RhcnQgYnkgdGFraW5nIG9uZSBtb250aCBvZiBjcmltZSBkYXRhLCBleHBsb3JpbmcgaXQsIGFuZCB3cml0aW5nIGFsbCBvdXIgc3RlcHMgZm9yIGF1dG9tYXRpb24uDQoNCmBgYHtyfQ0KdGVzdF9kZiA8LSByZWFkLmNzdigiY3JpbWVzLzIwMTgtMDEvMjAxOC0wMS1tZXRyb3BvbGl0YW4tc3RyZWV0LmNzdiIpDQp0ZXN0X2RmDQoNCmBgYA0KT3VyIGNyaW1lIGlzIGNhdGVnb3JpZXMgYWNjb3JkaW5nIHRvIHRoZSBIb21lIE9mZmljZSBtYWpvciBjcmltZSB0eXBlcywgYW5kIGxpa2UgUHl0aG9uLCB3ZSBjYW4gbGlzdCB0aGVtIGFsbCB0aHJvdWdoIHRoZSAidW5pcXVlIiBmdW5jdGlvbi4gSGVyZSBJJ2xsIGJlIGZvY3VzaW5nIG9uIHJvYmJlcnkgYW5kIGJ1cmdsYXJ5OiB0d28gY3JpbWUgdHlwZXMgdGhhdCBhcmUgaGVhdmlseSByZWxpYW50IG9uIGVuY291bnRlcmluZyB2aWN0aW0ncyBpbiBwdWJsaWMgc3BhY2VzLCBhbmQgYXMgc3VjaCBzaG91bGQgYmUgYWZmZWN0ZWQgYnkgdGhlICJDT1ZJRCBlZmZlY3QiLg0KDQpgYGB7cn0NCnVuaXF1ZSh0ZXN0X2RmWyJDcmltZS50eXBlIl0pDQpgYGANClRvIGF2b2lkIHRoaXMgZ2V0dGluZyBwYXJ0aWN1bGFybHkgY29tcHV0YXRpb25hbGx5IGludGVuc2l2ZSwgbGV0J3Mgd3JpdGUgYSBmdW5jdGlvbiB0byBwdWxsIG91dCByb2JiZXJpZXMgYW5kIGJ1cmdsYXJpZXMsIGFuZCBhc3NpZ24gdGhlbSBhIHNwZWNpZmljIE1TT0EuIFRoZW4gd2UgY2FuIGl0ZXJhdGUgb3ZlciBhbGwgb3VyIG1vbnRocyBhbmQgZ2V0IG1vbnRobHkgY291bnRzIGZvciBlYWNoIG9mZmVuY2UgdHlwZS4NCg0KYGBge3J9DQpzdWJzZXRfZGYgPC0gZmlsdGVyKHRlc3RfZGYsIENyaW1lLnR5cGU9PSJCdXJnbGFyeSIgfCBDcmltZS50eXBlPT0iUm9iYmVyeSIpDQpzdWJzZXRfZGYNCmBgYA0KT3VyIHNpbmdsZSBtb250aCBvZiBkYXRhIGNvbnRhaW5zIDEwLDUwMSBjcmltZXMuDQoNClRoZXJlIGFyZSBSL1B5dGhvbiBxdWlya3MgdGhhdCB3aWxsIHRha2Ugc29tZSBnZXR0aW5nIHVzZWQgdG8uIFdoaWxlIGluIFB5dGhvbiwgbW9zdCBjb2x1bW5zIGNhbiBiZSByZWZlcmVuY2VkIHRocm91Z2ggdGhlaXIgc3RyaW5nIG5hbWUgKFsibmFtZSJdKSwgUiBzZWVtcyBzb21ld2hhdCBmdXNzaWVyLiBPdGhlciB0aGFuIHRoYXQsIHRoZSBtb3ZlIGZyb20gb25lIHRvIGFub3RoZXIgaXMgbGFyZ2VseSBpbnR1aXRpdmUuDQoNCldlIG5vdyBuZWVkIHRvIGxpbmsgdGhpcyB0byBvdXIgc3BhdGlhbCBkYXRhLiBXZSB1c2UgdGhlIE1TT0EgYm9yZGVycyBwcm92aWRlZCBieSBNT1BBQywgYW5kIHVzZSB0aGUgVUsgTmF0aW9uYWwgR3JpZCBjb29yZGluYXRlIHN5c3RlbS4gUG9saWNlLnVrIGRvZXMgbm90IHVzZSB0aGF0IHN5c3RlbSwgc28gd2UnbGwgbmVlZCB0byByZXByb2plY3Qgb3VyIGNyaW1lIGRhdGEuDQoNCmBgYHtyfQ0KbHNvYV9ib3JkZXJzIDwtIHN0X3JlYWQoIm1zb2FfYm9yZGVycy9NU09BXzIwMTFfTG9uZG9uX2dlbl9NSFcudGFiIiwgY3JzPTI3NzAwKQ0KbHNvYV9ib3JkZXJzDQoNCmBgYA0KVW5saWtlIG91ciBwcmV2aW91cyBkYXRhZnJhbWVzLCB0aGlzIGlzbid0ICJ0aWR5IiAoZHVlIHRvIGEgc2xpZ2h0bHkgZGlmZmVyZW50IGZvcm1hdCBjb250YWluaW5nIGdlb2dyYXBoaWNhbCBkYXRhKQ0KDQpgYGB7cn0NCnBsb3QobHNvYV9ib3JkZXJzKQ0KYGBgDQpCZWZvcmUgd2UgY2FuIGxpbmsgb3VyIGNyaW1lcyB0byBNU09BLCB3ZSdsbCBuZWVkIHRvIGVuc3VyZSBpZGVudGljYWwgY29vcmRpbmF0ZSBzeXN0ZW1zLCBidXQgYmVmb3JlIHdlIGRvIHRoYXQsIHdlJ2xsIG5lZWQgdG8gZXJhc2UgYW55IG1pc3NpbmcgdmFsdWVzLg0KDQoNCmBgYHtyfQ0KI2NvdW50IG1pc3NpbmcgdmFsdWVzIGluIHRoZSBsb25naXR1ZGUgY29sdW1uDQpzdW0oaXMubmEoc3Vic2V0X2RmWyJMb25naXR1ZGUiXSkpDQpgYGANCg0KQXMgc3VjaCwgd2UgaGF2ZSA4MiBjcmltZXMgd2l0aCBubyBnZW9ncmFwaGljIGlkZW50aWZpZXJzLCB3aGljaCB3ZSByZW1vdmUgZnJvbSBvdXIgYW5hbHlzaXMuDQoNCmBgYHtyfQ0KY2xlYW5fZGYgPC0gc3Vic2V0X2RmWyFyb3dTdW1zKGlzLm5hKHN1YnNldF9kZlsiTG9uZ2l0dWRlIl0pKSwgXQ0KY2xlYW5fZGYNCmBgYA0KV2UgY2FuIG5vdyBjb252ZXJ0IG91ciBjcmltZSBkYXRhIHRvIHNwYWNpYWwgZGF0YSwgdXNpbmcgb3VyIGxvbmdpdHVkZSBhbmQgbGF0aXR1ZGUgY29vcmRpbmF0ZXMgLSB0aGlzIGFsbG93cyB1cyB0byBxdWlja2x5IHBsb3Qgb3VyIGRhdGEsIGFuZCBjb25maXJtIGl0IGxvb2tzIHJpZ2h0Lg0KDQpgYGB7cn0NCg0Kc3Vic2V0X3NwYXRpYWwgPC0gc3RfYXNfc2YoY2xlYW5fZGYsIGNvb3JkcyA9IGMoIkxvbmdpdHVkZSIsICJMYXRpdHVkZSIpLCANCiAgICAgICAgICAgICAgICAgICAgICBjcnMgPSA0MzI2LCByZW1vdmUgPSBGQUxTRSkNCg0Kc3Vic2V0X3NwYXRpYWwNCmBgYA0KDQoNCmBgYHtyfQ0KcGxvdChzdWJzZXRfc3BhdGlhbCkNCmBgYA0KU3VjY2VzcyEgVGhhdCBsb29rcyBmYWludGx5IHByb21pc2luZy4gIE5vdywgbGV0J3MgZmlndXJlIG91dCBob3cgdG8gcmUtcHJvamVjdC4NCg0KYGBge3J9DQoNCmxhdGxvbmcgPSAiK2luaXQ9ZXBzZzo0MzI2Ig0KdWtncmlkID0gIitpbml0PWVwc2c6Mjc3MDAiDQoNCmBgYA0KDQpgYGB7cn0NCnN1YnNldF9vc2diIDwtIHN0X3RyYW5zZm9ybShzdWJzZXRfc3BhdGlhbCwgdWtncmlkKQ0Kc3Vic2V0X29zZ2INCmBgYA0KDQpXZSBub3cgcnVuIGEgInNwYXRpYWwgam9pbiI6IGFzc2lnbmluZyBhbiBNU09BIHRvIGVhY2ggb2Ygb3VyIGNyaW1lcywgYmFzZWQgb24gdGhlIE1TT0EgYm9yZGVyIG1hcCB0aGV5IGFyZSB3aXRoaW4sIGFuZCBjb21iaW5pbmcgdGhlc2UgYXMgb25lIHRhYmxlLg0KDQpgYGB7cn0NCmNyaW1lX3dpdGhfbXNvYSA8LSBzdF9qb2luKHN1YnNldF9vc2diLCBsc29hX2JvcmRlcnNbIk1TT0ExMUNEIl0pDQpjcmltZV93aXRoX21zb2ENCmBgYA0KVGhhdCBoYXMgd29ya2VkOiBlYWNoIGNyaW1lIGlzIG5vdyBhc3NpZ25lZCB0byBhbiBNU09BLiBXZSBjYW4gbm93IGdyb3VwIG91ciB0YWJsZSBieSBjb3VudCBvZiBNU09BLCB0byBvYnRhaW4gdGhlIG1vbnRobHkgY291bnQgb2YNCg0KYGBge3J9DQptc29hX2xpc3Q8LSBjcmltZV93aXRoX21zb2EgJT4lDQogIGdyb3VwX2J5KE1TT0ExMUNELCBDcmltZS50eXBlKSAlPiUNCiAgc3VtbWFyaXplKGNvdW50X2J5X21zb2EgPSBuKCkpDQoNCm1zb2FfbGlzdA0KYGBgDQpXZSBub3cgaGF2ZSBhIGNvdW50IGZvciBlYWNoLCBhbmQgd2UgY2FuIHJlbW92ZSB0aGUgZ2VvbWV0cnkgZGF0YSBiZWZvcmUgYXV0b21hdGluZyBvdXIgcHJvY2Vzcy4NCg0KYGBge3J9DQptc29hX3Bpdm90X3RpYmJsZSA8LSBhc190aWJibGUobXNvYV9saXN0KQ0KbXNvYV9waXZvdF90aWJibGUgPC0gbXNvYV9waXZvdF90aWJibGVbMDozXQ0KbXNvYV9waXZvdF90aWJibGUNCg0KYGBgDQoNCkFzIGEgZmluYWwgc3RhZ2UgcHJpb3IgdG8gcHJvY2Vzc2luZyB0aGUgcmVzdCBvZiBvdXIgZGF0YSwgd2Ugd2lsbCBmaWxsIGFueSBtaXNzaW5nIG1zb2EvbW9udGggY29tYmluYXRpb25zIHdpdGggYSAiMCIsIHRvIGVuc3VyZSBjb25zaXN0ZW50IHRyZW5kcy4NCg0KYGBge3J9DQojY3JlYXRpbmcgYSBkZiB3aXRoIGFsbCBtc29hIG5hbWVzLCBmb3Igcm9iYmVyeSBhbmQgYnVyZ2xhcnkNCm1zb2FfemVyb19kZl9yb2JiZXJ5IDwtIHVuaXF1ZShhc190aWJibGUobHNvYV9ib3JkZXJzKVsiTVNPQTExQ0QiXSkNCm1zb2FfemVyb19kZl9idXJnbGFyeSA8LSB1bmlxdWUoYXNfdGliYmxlKGxzb2FfYm9yZGVycylbIk1TT0ExMUNEIl0pDQoNCiNhZGRpbmcgb3VyIGNyaW1lIHR5cGUgY29sdW1uIA0KbXNvYV96ZXJvX2RmX2J1cmdsYXJ5WyJDcmltZS50eXBlIl0gPSAiQnVyZ2xhcnkiDQptc29hX3plcm9fZGZfcm9iYmVyeVsiQ3JpbWUudHlwZSJdID0gIlJvYmJlcnkiDQoNCiNDcmVhdGluZyBhICJjb3VudCIgY29sdW1uIGlkZW50aWNhbCB0byBvdXIgcGl2b3QsIGFuZCBmaWxsaW5nIGl0IHdpdGggMA0KbXNvYV96ZXJvX2RmX2J1cmdsYXJ5WyJjb3VudF9ieV9tc29hIl0gPSBhcy5udW1lcmljKDApDQptc29hX3plcm9fZGZfcm9iYmVyeVsiY291bnRfYnlfbXNvYSJdID0gYXMubnVtZXJpYygwKQ0KDQojY29tYmluZSBib3RoDQpkdXBsaWNhdGVfY29uY2F0IDwtIHJiaW5kKG1zb2FfemVyb19kZl9yb2JiZXJ5LCBtc29hX3plcm9fZGZfYnVyZ2xhcnkpDQoNCiNhZGQgb3VyIGR1cGxpY2F0ZXMgdG8gb3VyIG9yaWdpbmFsIHRhYmxlDQpkZl93aXRoX2R1cHMgPC0gcmJpbmQobXNvYV9waXZvdF90aWJibGUsIGR1cGxpY2F0ZV9jb25jYXQpDQoNCg0KI2NyZWF0aW5nIGEgZmlsdGVyIGZvciBkdXBsaWNhdGVzIGNvbHVtbnMsIHdoaWNoIHNob3VsZCBpZ25vcmUgdGhlIGZpcnN0IGluc3RhbmNlDQpkdXBfZmlsdGVycyA8LSBkdXBsaWNhdGVkKGRmX3dpdGhfZHVwc1swOjJdKQ0KDQoNCiNicmluZyBpdCBhbGwgYmFjayB0b2dldGhlcg0KbW9udGhseV9kZiA8LSBmaWx0ZXIoZGZfd2l0aF9kdXBzLCAhZHVwX2ZpbHRlcnMpDQptb250aGx5X2RmDQoNCiNzZWxlY3QgdGhlIGZpcnN0IHVuaXF1ZSB2YWx1ZSBvZiBtb250aHMgaW4gdGhlIG9yaWdpbmFsIGRhdGFmcmFtZQ0KbW9udGggPC0gdW5pcXVlKHRlc3RfZGZbIk1vbnRoIl0pWzEsMV0NCm1vbnRobHlfZGZbIk1vbnRoIl0gPC0gbW9udGgNCm1vbnRobHlfZGYNCg0KYGBgDQp3ZSBub3cgaGF2ZSBvdXIgYmFzaWMgZnVuY3Rpb25hbGl0eSB0byBjcmVhdGUgYSB0aW1lIHNlcmllcyBmb3IgYm90aCBjcmltZSB0eXBlcywgYnkgTVNPQSAtIGEgY291bnQgb2YgZWFjaCB0eXBlLCBhbmQgdGhlIG1vbnRoLiANCg0KSSBjYW4gbm93IGJyaW5nIGFsbCBteSBwcmV2aW91cyB3b3JrIHRvZ2V0aGVyIGludG8gYSBmdW5jdGlvbiwgd2hpY2ggd2UgY2FuIHVzZSB0byBhdXRvbWF0ZSB0aGUgcHJvY2Vzcy4NCg0KYGBge3J9DQojcXVpY2sgaW5pdGlhbCBmdW5jdGlvbiB0byBnZW5lcmF0ZSBvdXIgTVNPQSBib3JkZSBzcGF0aWFsIGZyYW1lLCB0byBhdm9pZCBpdCBzaXR0aW5nIGluIHRoZSBpbml0aWFsIGZyYW1lIGFuZCBnb2JibGluZyBsb2FkcyBvZiBtZW1vcnkuDQpnZW5lcmF0ZV9tc29hX2JvcmRlcnMgPC0gZnVuY3Rpb24oZmlsZSl7DQogIG1zb2FfYm9yZGVycyA8LSBzdF9yZWFkKGZpbGUsIGNycz0yNzcwMCkNCiAgcmV0dXJuKG1zb2FfYm9yZGVycykNCn0NCg0KbWFrZV9tb250aF9waXZvdCA8LSBmdW5jdGlvbihmaWxlKXsNCiAgI2RlZmluZSBvdXIgQ1JTDQogIGxhdGxvbmcgPSAiK2luaXQ9ZXBzZzo0MzI2Ig0KICB1a2dyaWQgPSAiK2luaXQ9ZXBzZzoyNzcwMCINCiAgI3JlYWQgb3VyIGNyaW1lIGZyb20gdGhlIGZpbGUNCiAgdGVzdF9kZiA8LSByZWFkLmNzdihmaWxlKQ0KICAjc2VsZWN0IG9ubHkgb3VyIHRhcmdldCBjcmltZSB0eXBlcw0KICBzdWJzZXRfZGYgPC0gZmlsdGVyKHRlc3RfZGYsIENyaW1lLnR5cGU9PSJCdXJnbGFyeSIgfCBDcmltZS50eXBlPT0iUm9iYmVyeSIpDQogICNyZW1vdmUgYW55IHJvd3Mgd2l0aCBhIGxvbmcvbGF0IGNvb3JkaW5hdGUNCiAgY2xlYW5fZGYgPC0gc3Vic2V0X2RmWyFyb3dTdW1zKGlzLm5hKHN1YnNldF9kZlsiTG9uZ2l0dWRlIl0pKSwgXQ0KICAjZ2VuZXJhdGUgYSBzcGF0aWFsIGRmDQogIHN1YnNldF9zcGF0aWFsIDwtIHN0X2FzX3NmKGNsZWFuX2RmLCBjb29yZHMgPSBjKCJMb25naXR1ZGUiLCAiTGF0aXR1ZGUiKSwgDQogICAgICAgICAgICAgICAgICAgICAgY3JzID0gNDMyNiwgcmVtb3ZlID0gRkFMU0UpDQogICNyZXByb2plY3QgdG8gdWsgZ3JpZCBjb29yZHMNCiAgc3Vic2V0X29zZ2IgPC0gc3RfdHJhbnNmb3JtKHN1YnNldF9zcGF0aWFsLCB1a2dyaWQpDQogICNzcGF0aWFsbHkgam9pbiB0byBhc3NpZ24gdG8gYW4gTVNPQQ0KICBjcmltZV93aXRoX21zb2EgPC0gc3Rfam9pbihzdWJzZXRfb3NnYiwgbXNvYV9ib3JkZXJzWyJNU09BMTFDRCJdKQ0KICAjc3VtbWFyaXNlIGJ5IGNvdW50IG9mIE1TT0ENCiAgbXNvYV9saXN0PC0gY3JpbWVfd2l0aF9tc29hICU+JQ0KICAgIGdyb3VwX2J5KE1TT0ExMUNELCBDcmltZS50eXBlKSAlPiUNCiAgICBzdW1tYXJpemUoY291bnRfYnlfbXNvYSA9IG4oKSkNCiAgI3JldHVybiB0byBhIG5vbi1nZW9ncmFwaGljIG1zb2ENCiAgbXNvYV9waXZvdF90aWJibGUgPC0gYXNfdGliYmxlKG1zb2FfbGlzdCkNCiAgbXNvYV9waXZvdF90aWJibGUgPC0gbXNvYV9waXZvdF90aWJibGVbMDozXQ0KICAjY3JlYXRpbmcgYSBkZiB3aXRoIGFsbCBtc29hIG5hbWVzLCBmb3Igcm9iYmVyeSBhbmQgYnVyZ2xhcnkNCiAgbXNvYV96ZXJvX2RmX3JvYmJlcnkgPC0gdW5pcXVlKGFzX3RpYmJsZShtc29hX2JvcmRlcnMpWyJNU09BMTFDRCJdKQ0KICBtc29hX3plcm9fZGZfYnVyZ2xhcnkgPC0gdW5pcXVlKGFzX3RpYmJsZShtc29hX2JvcmRlcnMpWyJNU09BMTFDRCJdKQ0KICAjYWRkaW5nIG91ciBjcmltZSB0eXBlIGNvbHVtbiANCiAgbXNvYV96ZXJvX2RmX2J1cmdsYXJ5WyJDcmltZS50eXBlIl0gPSAiQnVyZ2xhcnkiDQogIG1zb2FfemVyb19kZl9yb2JiZXJ5WyJDcmltZS50eXBlIl0gPSAiUm9iYmVyeSINCiAgI0NyZWF0aW5nIGEgImNvdW50IiBjb2x1bW4gaWRlbnRpY2FsIHRvIG91ciBwaXZvdCwgYW5kIGZpbGxpbmcgaXQgd2l0aCAwDQogIG1zb2FfemVyb19kZl9idXJnbGFyeVsiY291bnRfYnlfbXNvYSJdID0gYXMubnVtZXJpYygwKQ0KICBtc29hX3plcm9fZGZfcm9iYmVyeVsiY291bnRfYnlfbXNvYSJdID0gYXMubnVtZXJpYygwKQ0KICBkdXBsaWNhdGVfY29uY2F0IDwtIHJiaW5kKG1zb2FfemVyb19kZl9yb2JiZXJ5LCBtc29hX3plcm9fZGZfYnVyZ2xhcnkpDQogIGRmX3dpdGhfZHVwcyA8LSByYmluZChtc29hX3Bpdm90X3RpYmJsZSwgZHVwbGljYXRlX2NvbmNhdCkNCiAgI2NyZWF0aW5nIGEgZmlsdGVyIGZvciBkdXBsaWNhdGVzIGNvbHVtbnMsIHdoaWNoIHNob3VsZCBpZ25vcmUgdGhlIGZpcnN0IGluc3RhbmNlDQogIGR1cF9maWx0ZXJzIDwtIGR1cGxpY2F0ZWQoZGZfd2l0aF9kdXBzWzA6Ml0pDQogIG1vbnRobHlfZGYgPC0gZmlsdGVyKGRmX3dpdGhfZHVwcywgIWR1cF9maWx0ZXJzKQ0KICAjcmUtYWRkIG91ciBtb250aCBjb2x1bW4NCiAgbW9udGggPC0gdW5pcXVlKHRlc3RfZGZbIk1vbnRoIl0pWzEsMV0NCiAgbW9udGhseV9kZlsiTW9udGgiXSA8LSBtb250aA0KICByZXR1cm4obW9udGhseV9kZikNCn0NCmBgYA0KDQpXZSBub3cgaGF2ZSBhIHJ1ZGltZW50YXJ5IHRvb2xpbmcgcGlwZWxpbmUsIHdoaWNoIHdlIGNhbiBpdGVyYXRlIG92ZXIgb3VyIHN1YmZvbGRlcnMgdG8gYWdncmVnYXRlIG91ciBkYXRhIChpbiBoaW5kc2V0LCBJIHByb2JhYmx5IHNob3VsZCBoYXZlIGV4cGxvcmVkIHRoZSBBUEkgb3B0aW9ucykuDQoNCiAgDQpgYGB7ciwgcmVzdWx0cz0naGlkZSd9DQojY3JlYXRlIGVtcHR5IGRhdGFmcmFtZSB0byBicmluZyB0b2dldGhlciBvdXIgZGF0YQ0KZW1wdHlfZGYgPC0gdGliYmxlKA0KTVNPQTExQ0QgPSAiIiwgDQpDcmltZS50eXBlPSAiIiwNCmNvdW50X2J5X21zb2E9ICIiLA0KTW9udGg9ICIiDQopDQoNCiNyZS1pbmdlc3Qgb3VyIE1TT0EgZGF0YQ0KbXNvYV9ib3JkZXJzIDwtIGdlbmVyYXRlX21zb2FfYm9yZGVycygibXNvYV9ib3JkZXJzL01TT0FfMjAxMV9Mb25kb25fZ2VuX01IVy50YWIiKQ0KYGBgDQoNCk91ciBmdW5jdGlvbiB3aWxsIGl0ZXJhdGUgb3ZlciBlYWNoIGZpbGUsIGFkZCB0aGVtIHRvIG91ciBlbXB0eSB0YWJsZSwgdW50aWwgZWFjaCBpcyBjb21wbGV0ZSAoYW5kIHdlIGhhdmUgYSBzaW5nbGUsIGFnZ3JlZ2F0ZWQgZmlsZSkNCg0KYGBge3IsIHJlc3VsdHM9J2hpZGUnfQ0Kc3ViZmlsZXMgPC0gbGlzdC5maWxlcyhwYXRoID0gImNyaW1lcyIsIHJlY3Vyc2l2ZT1UKQ0KDQpmb3IgKGZpbGUgaW4gc3ViZmlsZXMpew0KICBmb2xkZXJfc3ViZGlyIDwtICJjcmltZXMvIg0KICAjY29uY2F0ZW5hdGUgdG8gZ2V0IG91ciB0b3RhbCBzdWJmb2xkZXIgZGlyZWN0b3J5IC0gaGFja3kgYnV0IHdpbGwgd29yayBoZXJlLg0KICBzdWJfcGF0aCA8LSBwYXN0ZShmb2xkZXJfc3ViZGlyLCBmaWxlLCBzZXA9IiIpDQogIG1vbnRobHlfZGYgPC0gbWFrZV9tb250aF9waXZvdChzdWJfcGF0aCkNCiAgZW1wdHlfZGYgPC0gcmJpbmQoZW1wdHlfZGYsIG1vbnRobHlfZGYpDQp9DQoNCmBgYA0KDQoNCmBgYHtyfQ0KZW1wdHlfZGYNCmBgYA0KV2Ugbm93IGhhdmUgYSBjb21iaW5lZCBkYXRhZnJhbWUgb2YgNzEsODQ4IHJvd3MsIGZyb20gSmFudWFyeSAyMDE4IHRocm91Z2ggRGVjZW1iZXIgMjAyMC4NCg0KYGBge3J9DQp1bmlxdWUoZW1wdHlfZGZbIk1vbnRoIl0pDQpgYGANCkZyb20gYSBwcmFjdGljYWwgcGVyc3BlY3RpdmUsIHRoYXQgd2FzIHNsaWdodGx5IG1vcmUgcGFpbmZ1bCB0aGFuIEkgZXhwZWN0ZWQgLSBJJ20gbm90IHN1cmUgaWYgZGF0YS13cmFuZ2xpbmcgaXMgbGVzcyBpbnR1aXRpdmUgaW4gUiwgb3IgaXQncyBqdXN0IHByYWN0aWNlLCBidXQgaXQncyBub3RpY2VhYmxlIGhvdyBlYXN5IHVzZSBjYXNlcyBhcmUgbGVzcyBlYXNpbHkgYWNjZXNzaWJsZSBhcyB0dXRvcmlhbHMgKHByb2JhYmx5IGR1ZSB0byB0aGUgZGlmZmVyZW50IHVzZXIgYmFzZQ0KKQ0KDQoNCmBgYHtyfQ0KI3NhdmluZyBmaWxlIHRvIENTVg0KI3dyaXRlLmNzdihlbXB0eV9kZiwibXNvYV9jcmltZV9tYXRyaXguY3N2IikNCg0KYGBgDQoNCg0KDQojIyAyLiBQcmVkaWN0IHRyZW5kIGJ5IE1TT0ENCiMjIyMgVmlzdWFsaXNhdGlvbiBhbmQgRXhwbG9yYXRpb24NCldpdGggb3VyIGRhdGEgbm93IGNsZWFuZWQgYW5kIGFnZ3JlZ2F0ZWQsIHdlIGNhbiBmb2N1cyBvbiB0aGUgbW9yZSBpbnRlcmVzdGluZyBwYXJ0IC0gZm9yZWNhc3Rpbmcgb3VyICJleHBlY3RlZCIgcGFuZGVtaWMgY3JpbWUsIGFuZCBleGFtaW5pbmcgaG93IG11Y2ggaXQgZGl2ZXJnZXMgZnJvbSBvdXIgImFjdHVhbCIgY3JpbWUuDQoNCmBgYHtyfQ0KZW1wdHlfZGYgPC0gcmVhZC5jc3YoIm1zb2FfY3JpbWVfbWF0cml4LmNzdiIpDQplbXB0eV9kZiA8LSBlbXB0eV9kZlsyOjcwODQ4LDI6NV0NCmVtcHR5X2RmDQpgYGANCg0KDQpCZWZvcmUgZ29pbmcgYW55IGZ1cnRoZXIsIGxldCdzIHVzZSB0aGlzIHRvIGV4cGxvcmUgYW5kIHZpc3VhbGlzZSB0aGUgZGlzdHJpYnV0aW9uIG9mIHJvYmJlcnkgYW5kIGJ1cmdsYXJ5IGFjcm9zcyB0aW1lIGFuZCBzcGFjZSBkdXJpbmcgb3VyICJwcmUtcGFuZGVtaWMiIHBlcmlvZCwgaW4gTWFyY2ggMjAyMCAtIGJhc2VkIG9uIExvbmRvbiBtb2JpbGl0eSBpbmRpY2F0b3JzLCB0aGlzIGlzIHdoZW4gbW92ZW1lbnQgYWNjcm9zcyBMb25kb24gYmVnYW4gdG8gYmUgaGVhdmlseSBhZmZlY3RlZCwgYW5kIHRoZSBkaXNydXB0aW9uIHdhcyBtb3N0IG5vdGFibGUgaW4gQXByaWwNCg0KIVtMb25kb24gbW9iaWxpdHkgZGF0YV0obG9uZG9uX21vYmlsaXR5LnBuZykNCg0KYGBge3J9DQpidXJnbGFyeV9kZjwtZW1wdHlfZGYNCg0KI2FkZCBhICIxIiBzbyBvdXIgbW9udGggY2FuIGJlIGNvbnZlcnRlZCB0byBhIGZ1bGwgZGF0ZQ0KYnVyZ2xhcnlfZGYkRGF0ZVN0cmluZyA8LSBwYXN0ZShidXJnbGFyeV9kZiRNb250aCwgIi0wMSIsIHNlcD0iIikNCg0KI2NvbnZlcnQgdG8gZGF0ZSBmb3JtYXQNCmJ1cmdsYXJ5X2RmJERhdGVDbGVhbiA8LSB5bWQoYnVyZ2xhcnlfZGYkRGF0ZVN0cmluZykNCg0KI2ZpbHRlciBvdXQgb25seSBidXJnbGFyeSBwcmlvciB0byB0aGUgcGFuZGVtaWMNCmJ1cmdsYXJ5RXhwbG9yZSA8LSBmaWx0ZXIoYnVyZ2xhcnlfZGYsICBEYXRlQ2xlYW4gPCAiMjAyMC0wMy0wMSIgJiBDcmltZS50eXBlPT0iQnVyZ2xhcnkiKQ0KDQpidXJnbGFyeUV4cGxvcmUNCmBgYA0KDQoNCkxvb2tpbmcgYXQgdGhlIGFnZ3JlZ2F0ZSBjb3VudHMgb2YgYnVyZ2xhcnkgYWNyb3NzIExvbmRvbiwgYSB2aXN1YWwgb2JzZXJ2YXRpb24gc3VnZ2VzdHMgeWVhcmx5IHRyZW5kcyAod2hpY2ggd2UnbGwgaGF2ZSB0byBjb25zaWRlciBpbiBvdXIgZm9yZWNhc3QpLCB3aGljaCBzaGFycCBwZWFrcyBkdXJpbmcgdGhlIFdpbnRlciBtb250aHMgYW5kIHRoZSBsb3dlc3QgbnVtYmVycyBpbiBzdW1tZXIgKHdoZW4gdGhlIGRheXMgYXJlIGxvbmdlc3QpLg0KYGBge3J9DQojZ3JvdXAgYnVyZ2xhcnkgY291bnQgYnkgbW9udGhzIGFuZCBwbG90DQpidXJnbGFyeV9ieV9tb250aCA8LSBidXJnbGFyeUV4cGxvcmUgJT4lDQogIGdyb3VwX2J5KERhdGVDbGVhbikgJT4lDQogIHN1bW1hcml6ZSh0b3RhbF9idXJnbGFyaWVzID0gc3VtKGNvdW50X2J5X21zb2EpKQ0KDQpnZ3Bsb3QoYnVyZ2xhcnlfYnlfbW9udGgsIGFlcyh4PURhdGVDbGVhbiwgeT10b3RhbF9idXJnbGFyaWVzKSkgKw0KICBnZW9tX2xpbmUoKQ0KYGBgDQoNClRvIG9ic2VydmUgaG93IGNyaW1lIGNvdW50cyBhcmUgZGlzdHJpYnV0ZWQgaW4gc3BhY2UsIGxldCdzIG1hcCBib3RoIGNvdW50cyBieSBNU09BLiBBcyAgcHJldmlvdXNseSBtZW50aW9uZWQsIE1TT0FzIGFyZSBkZXNpZ25lZCB0byBiZSBjb21wYXJhYmxlIHVuaXRzLCBhdCBsZWFzdCBmcm9tIGEgcG9wdWxhdGlvbiBwZXJzcGVjdGl2ZSAtIHdlIGRvbid0IG5lZWQgdG8gcHJvZHVjZSBwZXIgcG9wdWxhdGlvbiByYXRlcy4gDQoNCg0KYGBge3IsIGZpZy53aWR0aCA9IDEzfQ0KYnVyZ2xhcnlfYnlfbXNvYSA8LSBidXJnbGFyeUV4cGxvcmUgJT4lDQogIGdyb3VwX2J5KE1TT0ExMUNEKSAlPiUNCiAgc3VtbWFyaXplKHRvdGFsX2J1cmdsYXJpZXMgPSBzdW0oY291bnRfYnlfbXNvYSkpDQoNCiN3ZSBqb2luIG91ciBidXJnbGFyeSBjb3VudHMgdG8gdGhlaXIgZ2VvZ3JhcGhpYyBtc29hDQpidXJnbGFyeV9tYXAgPC0gbGVmdF9qb2luKGxzb2FfYm9yZGVycywgYnVyZ2xhcnlfYnlfbXNvYSwgYnkgPSAiTVNPQTExQ0QiKQ0KDQojdXNlciBicmV3ZXIgY29sb3VyIHBhbGV0dGUgaHR0cHM6Ly9jb2xvcmJyZXdlcjIub3JnDQpwYWwgPC0gYnJld2VyLnBhbCg1LCJCdUduIikNCg0KI2NyZWF0ZSBvdXIgbWFwLCBhbmQgYWRkIHRoZSBsYXlvdXQgb3B0aW9ucw0KYnVyZ2xhcnlfbWFwIDwtdG1fc2hhcGUoYnVyZ2xhcnlfbWFwKSArDQogIHRtX2ZpbGwoY29sID0gInRvdGFsX2J1cmdsYXJpZXMiLCB0aXRsZSA9ICJUb3RhbCBCdXJnbGFyeSBDb3VudCBieSBNU09BIiwgc3R5bGU9InF1YW50aWxlIiwgcGFsZXR0ZT0iQnVHbiIpICsNCiAgdG1fbGF5b3V0KGxlZ2VuZC5vdXRzaWRlID0gVFJVRSwgbGVnZW5kLm91dHNpZGUucG9zaXRpb24gPSAicmlnaHQiKQ0KDQpyb2JiZXJ5X2RmPC1lbXB0eV9kZg0KDQpyb2JiZXJ5X2RmJERhdGVTdHJpbmcgPC0gcGFzdGUocm9iYmVyeV9kZiRNb250aCwgIi0wMSIsIHNlcD0iIikNCnJvYmJlcnlfZGYkRGF0ZUNsZWFuIDwtIHltZChyb2JiZXJ5X2RmJERhdGVTdHJpbmcpDQpyb2JiZXJ5RXhwbG9yZSA8LSBmaWx0ZXIocm9iYmVyeV9kZiwgIERhdGVDbGVhbiA8ICIyMDIwLTAzLTAxIiAmIENyaW1lLnR5cGU9PSJSb2JiZXJ5IikNCg0Kcm9iYmVyeV9ieV9tc29hIDwtIHJvYmJlcnlFeHBsb3JlICU+JQ0KICBncm91cF9ieShNU09BMTFDRCkgJT4lDQogIHN1bW1hcml6ZSh0b3RhbF9yb2JiZXJpZXMgPSBzdW0oY291bnRfYnlfbXNvYSkpDQoNCnJvYmJlcnlfbWFwIDwtIGxlZnRfam9pbihsc29hX2JvcmRlcnMsIHJvYmJlcnlfYnlfbXNvYSwgYnkgPSAiTVNPQTExQ0QiKQ0KDQpwYWwgPC0gYnJld2VyLnBhbCg1LCJCdUduIikNCg0KDQpyb2JiZXJ5X21hcCA8LXRtX3NoYXBlKHJvYmJlcnlfbWFwKSArDQogIHRtX2ZpbGwoY29sID0gInRvdGFsX3JvYmJlcmllcyIsIHRpdGxlID0gIlRvdGFsIFJvYmJlcnkgQ291bnQgYnkgTVNPQSIsIHN0eWxlPSJxdWFudGlsZSIsIHBhbGV0dGU9IkJ1R24iKSArDQogIHRtX2xheW91dChsZWdlbmQub3V0c2lkZSA9IFRSVUUsIGxlZ2VuZC5vdXRzaWRlLnBvc2l0aW9uID0gInJpZ2h0IikNCg0KDQojYXJyYW5nZSB0aGUgbWFwcyB0b2dldGhlcg0KdG1hcF9hcnJhbmdlKGJ1cmdsYXJ5X21hcCwgcm9iYmVyeV9tYXAsIG5yb3cgPSAyKQ0KYGBgDQoNCldlIG5vdGljZSB0aGF0IHJvYmJlcnkgaXMgbm90aWNlYWJseSBtb3JlIGNvbmNlbnRyYXRlZCBpbiBjZW50cmFsIExvbmRvbiwgd2l0aCBidXJnbGFyeSByZW1haW5pbmcgcXVpdGUgY29tbW9uIGFjcm9zcyB0aGUgY2l0eS4gVGhhdCBzYWlkLCB0aGVyZSBhcmUgYWxzbyBvYnZpb3VzIHNwYXRpYWwgcGF0dGVybnMgaGVyZSAtIHRoZXNlIGNyaW1lcyBhcmUgY2x1c3RlcmVkIGluIGNlcnRhaW4gZ2VvZ3JhcGhpZXMuIA0KDQojIyMjIE1vZGVsbGluZyANCldlIGNhbiBub3cgYmVnaW4gdGhlIGZvcmVjYXN0aW5nIHByb2Nlc3MuIFRvIGRlc2lnbiBvdXIgcHJvY2Vzcywgd2UnbGwgc3RhcnQgYnkgZm9jdXNpbmcgb24gYSBzaW5nbGUgTVNPQSAtIHRoZSBmaXJzdCBpbiBvdXIgZGF0YXNldCwgW0UwMjAwMDAwMSwgb3IgdGhlIENpdHkgb2YgTG9uZG9uLl0oaHR0cHM6Ly9maW5kdGhhdHBvc3Rjb2RlLnVrL2FyZWFzL0UwMjAwMDAwMS5odG1sKQ0KDQpgYGB7cn0NCnNpbmdsZV9tc29hX2RmIDwtIGZpbHRlcihlbXB0eV9kZiwgTVNPQTExQ0QgPT0gIkUwMjAwMDAwMSIgJiBDcmltZS50eXBlPT0iQnVyZ2xhcnkiKQ0KDQojd2UgYWRkIGEgMDEgdG8gb3VyIGRhdGUgdG8gZW5zdXJlIFIgcmVjb2duaXNlcyB0aGUgZGF0ZSBmb3JtYXQNCnNpbmdsZV9tc29hX2RmJERhdGVTdHJpbmcgPC0gcGFzdGUoc2luZ2xlX21zb2FfZGYkTW9udGgsICItMDEiKQ0KDQoNCnNpbmdsZV9tc29hX2RmJERhdGVDbGVhbiA8LSB5bWQoc2luZ2xlX21zb2FfZGYkRGF0ZVN0cmluZykNCnNpbmdsZV9tc29hX2RmDQpgYGANCkZyb20gYSBmb3JlY2FzdGluZy90aW1lLXNlcmllcyBwZXJzcGVjdGl2ZSwgdGhpcyBpcyBhIHZlcnkgc21hbGwgZGF0YXNldCAtIDM2IG1vbnRobHkgb2JzZXJ2YXRpb25zLiBXZSB3aWxsIGJlIHNocmlua2luZyB0aGlzIGZ1cnRoZXIgdG8gb25seSAyNiBieSBmb2N1c2luZyBvbiBkYXRhIHByaW9yIHRvIE1hcmNoIDIwMjAsIHdoZW4gdGhlIENPVklEIGNyaW1lIGltcGFjdCBpcyBmZWx0LiBUaGlzIHNpZ25pZmljYW50bHkgbGltaXRzIG91ciBmb3JlY2FzdGluZyBvcHRpb25zLCBhbmQgd2lsbCBpbXBhY3QgYWNjdXJhY3ksIGlmIHdlIHRyZWF0IGVhY2ggTVNPQSBpbiBpc29sYXRpb24gLSB3ZSBjb3VsZCBleHBsb3JlIHNvbWUgc29ydCBvZiBWZWN0b3IgQXV0b3JlZ3Jlc3NpdmUgTW9kZWwgdG8gbGltaXQgdGhpcywgYnV0IGdpdmVuIHRoYXQgd2UncmUgdGhlbiBnb2luZyB0byBiZSBleHBsb3JpbmcgdGhlIGVycm9yIG9mIGFsbCBvdXIgbW9kZWxzIGluIGFnZ3JlZ2F0aW9uLCB0aGlzIGlzbid0IGNydWNpYWwuIE91ciBmb2N1cyBpcyBvbiBtb2RlbHMgdGhhdCB3ZSBjYW4gYWNjdXJhdGVseSBkZXBsb3kgd2l0aG91dCBuZWVkaW5nIHRvIHR1bmUgZWFjaCBvZiB0aGVtIGluZGl2aWR1YWxseSwgYW5kIHRoYXQgY2FuIGNhcHR1cmUgdGhlIHNlYXNvbmFsIHRyZW5kLCBhbmQgZ2VuZXJhdGUgcmVsaWFibGUgcHJlZGljdGlvbnMgb24gb3VyIGxpbWl0ZWQgZGF0YXNldC4gDQoNCkdpdmVuIHRoZXNlIGxpbWl0YXRpb25zLCBJJ3ZlIG9wdGVkIGZvciBbdGhlIFByb3BoZXQgYWxnb3JpdGguXShodHRwczovL2ZhY2Vib29rLmdpdGh1Yi5pby9wcm9waGV0LykgV2hpbGUgaXQncyBtb3JlIG9wYXF1ZSB0aGFuIGEgYXV0by1hcmltYSBvciBWQVIgbW9kZWwsIGl0IHdvcmtzIHdlbGwgd2l0aCBtb250aGx5IGRhdGEsIGFuZCBleHRyYWN0aW5nIHNlYXNvbmFsIHRyZW5kcy4gSXQgYWxzbyByZXF1aXJlcyB2ZXJ5IGxpdHRsZSB0dW5pbmcuDQoNCkFzIHN1Y2gsIHdlJ2xsIGV4dHJhY3Qgb3VyICJ0cmFpbmluZyBzZXQiIHByaW9yIHRvIE1hcmNoLCBhbmQgc3RhcnQgZm9yZWNhc3RpbmcuDQoNCmBgYHtyfQ0KDQp0cmFpbmluZ19zZXQgPC0gZmlsdGVyKHNpbmdsZV9tc29hX2RmLCBEYXRlQ2xlYW4gPCAiMjAyMC0wMy0wMSIpDQoNCnRyYWluaW5nX2RmIDwtIHRpYmJsZSgNCiAgZHM9dHJhaW5pbmdfc2V0JERhdGVDbGVhbiwNCiAgeT10cmFpbmluZ19zZXQkY291bnRfYnlfbXNvYQ0KKQ0KdHJhaW5pbmdfZGYNCmBgYA0KDQpgYGB7ciwgcmVzdWx0cz0naGlkZSd9DQpsaWJyYXJ5KHByb3BoZXQpDQptIDwtIHByb3BoZXQodHJhaW5pbmdfZGYpDQoNCmBgYA0KRm9yIG5vdywgd2UnbGwgZm9yZWNhc3Qgb24gYSA2IG1vbnRoIGhvcml6b24gLSB3ZSBvYnZpb3VzbHkgd291bGRuJ3QgZXhwZWN0IGl0IHRvIGJlIGFjY3VyYXRlIHRoYXQgZmFyIGludG8gdGhlIGZ1dHVyZS4NCg0KYGBge3J9DQojcHJvcGhldCBnZW5lcmF0ZXMgYSBmdXR1cmUgZGF0YWZyYW1lIHVzaW5nIG91ciBkYXRhLCBmb3IgNiBtcGVyaW9kcw0KZnV0dXJlIDwtIG1ha2VfZnV0dXJlX2RhdGFmcmFtZShtLCBwZXJpb2RzID0gNiwgZnJlcSA9ICdtb250aCcpDQoNCg0KZm9yZWNhc3QgPC0gcHJlZGljdChtLCBmdXR1cmUpDQoNCnBsb3QobSwgZm9yZWNhc3QpDQoNCmBgYA0KQXMgd2UgY2FuIHNlZSwgdGhlIG1vZGVsIHNlZW1zIGNvbnNpc3RlbnQgb24gYSBzaG9ydCBob3Jpem9uLCBhbmQgZ2V0cyB2ZXJ5IHdpZGUgYXMgaXQgZ29lcyBmdXJ0aGVyIGludG8gdGhlIGZ1dHVyZS4gTW9yZSBpbXBvcnRhbnRseSBob3dldmVyLCBpdCBoYXMgZXh0cmFjdGVkIGEgeWVhcmx5IHNlYXNvbmFsIGNvbXBvbnRlbnQgLSB0aGUgc3VtbWVyIGRlY3JlYXNlIHdlIGlkZW50aWZpZWQgcHJldmlvdXNseSAtIGFzIHdlbGwgYXMgYSBsb25nIHRlcm0gdHJlbmQuICANCg0KYGBge3J9DQoNCnByb3BoZXRfcGxvdF9jb21wb25lbnRzKG0sIGZvcmVjYXN0KQ0KYGBgDQpUaGVzZSBwcmVkaWN0aW9ucyBzZWVtIGZhci1mZXRjaGVkLCBidXQgcmVtZW1iZXIgd2Ugd2lsbCBiZSBvYnNlcnZpbmcgYSBMb25kb24gd2lkZSBlcnJvciByYXRlLiBBcyBzdWNoLCB3ZSBtdXN0IG5vdyBpc29sYXRlIG91ciAicGFuZGVtaWMgcGVyaW9kIiAtIHdoaWNoIHdlIGRlZmluZSBhcyBBcHJpbCBhbmQgTWF5IDIwMjAgLSBhbmQgY29tcGFyZSB0aGUgcHJlZGljdGVkIGNyaW1lIGNvdW50cyB0byB0aGUgYWN0dWFsIGNyaW1lIGNvdW50cyB0byBvYnRhaW4gYSBtZXRyaWMgb2Ygb3VyICJDT1ZJRCBjcmltZSBzaGlmdCIsIG9yIG91ciBlcnJvciByYXRlLg0KDQpgYGB7cn0NCg0KDQpmb3JlY2FzdCRNb250aCA8LSBtb250aChmb3JlY2FzdCRkcykNCmZvcmVjYXN0JFllYXIgPC0geWVhcihmb3JlY2FzdCRkcykNCg0KDQp0aGlzX3llYXIgPC0gZmlsdGVyKGZvcmVjYXN0LCBZZWFyID4gMjAxOSkNCnBlYWtfcGFuZGVtaWMgPC0gZmlsdGVyKHRoaXNfeWVhciwgTW9udGg9PSA0IHwgTW9udGg9PSA1ICkNCg0KcHJlZGljdGlvblBpdm90IDwtIHBlYWtfcGFuZGVtaWMgJT4lDQogIGdyb3VwX2J5KE1vbnRoKSAlPiUNCiAgc3VtbWFyaXplKHByZWRpY3RlZF9idXJnbGFyeSA9IG1lYW4oeWhhdCkpDQoNCg0Kc2luZ2xlX21zb2FfZGYkTW9udGhOdW0gPC0gbW9udGgoc2luZ2xlX21zb2FfZGYkRGF0ZUNsZWFuKQ0Kc2luZ2xlX21zb2FfZGYkWWVhck51bSA8LSB5ZWFyKHNpbmdsZV9tc29hX2RmJERhdGVDbGVhbikNCg0KdGhpc195ZWFyX2FjdHVhbCA8LSBmaWx0ZXIoc2luZ2xlX21zb2FfZGYsIFllYXJOdW0gPiAyMDE5KQ0KcGVha19wYW5kZW1pY19hY3R1YWwgPC0gZmlsdGVyKHRoaXNfeWVhcl9hY3R1YWwsIE1vbnRoTnVtPT0gNCB8IE1vbnRoTnVtPT0gNSApDQoNCmFjdHVhbF9idXJnbGFyeSA8LSBzdW0ocGVha19wYW5kZW1pY19hY3R1YWwkY291bnRfYnlfbXNvYSkNCnByZWRfYnVyZ2xhcnkgPC0gc3VtKHByZWRpY3Rpb25QaXZvdCRwcmVkaWN0ZWRfYnVyZ2xhcnkpDQoNCmVycm9yIDwtIGFjdHVhbF9idXJnbGFyeSAtIHByZWRfYnVyZ2xhcnkNCnBlcmNlbnRhZ2VfZXJyb3IgPC0gZXJyb3IgLyBwcmVkX2J1cmdsYXJ5IA0KDQpwcmludCgiQnVyZ2xhcnkgQ291bnQiKQ0KcHJpbnQoYWN0dWFsX2J1cmdsYXJ5KQ0KcHJpbnQoIlByZWRpY3RlZCIpDQpwcmludChwcmVkX2J1cmdsYXJ5KQ0KDQpwcmludCgiQWN0dWFsIEVycm9yIikNCnByaW50KGVycm9yKQ0KcHJpbnQoIlBlcmNlbnRhZ2UgRXJyb3IiKQ0KcHJpbnQocGVyY2VudGFnZV9lcnJvcikNCmBgYA0KSW4gdGhpcyBNU09BLCBvdXIgbW9kZWwgcHJlZGljdGVkIG5lYXJseSA4IGJ1cmdsYXJpZXMgd291bGQgb2NjdXIgaW4gdGhlc2UgdHdvIG1vbnRocywgYmFzZWQgb24gcHJlLXBhbmRlbWljIHRyZW5kcy4gSW4gcmVhbGl0eSwgMSB0b29rIHBsYWNlIC0gYSBsYXJnZSBlcnJvciByYXRlLCBzdWdnZXN0aW5nIGEgc3Ryb25nICJDT1ZJRCBlZmZlY3QiLg0KDQpUaGlzIHByb2Nlc3MgY2FuIG5vdyBiZSByZXBsaWNhdGVkIGZvciBldmVyeSBNU09BIGluIExvbmRvbiwgdG8gb2J0YWluIHRoaXMgbWV0cmljIGZvciBlYWNoIE1TT0EuDQoNCmBgYHtyLCByZXN1bHRzPSdoaWRlJ30NCg0KDQptc29hX2Vycm9yX3RpYmJsZSA8LSB0aWJibGUoDQpNU09BMTFDRCA9ICIiLCANCmJ1cmdsYXJ5QWN0dWFsPSAiIiwNCmJ1cmdsYXJ5UHJlZGljdGVkPSAiIiwNCmJ1cmdsYXJ5RXJyb3I9ICIiLA0KYnVyZ2xhcnlQZXJjZW50RXJyb3I9IiIsDQpyb2JiZXJ5QWN0dWFsPSAiIiwNCnJvYmJlcnlQcmVkaWN0ZWQ9ICIiLA0Kcm9iYmVyeUVycm9yPSAiIiwNCnJvYmJlcnlQZXJjZW50RXJyb3I9IiINCikNCg0KY2FsY3VsYXRlX2Vycm9yIDwtIGZ1bmN0aW9uKG1zb2FOYW1lKXsNCiAgI3NlbGVjdCBvbmx5IGJ1cmdsYXJ5IGFuZCBvdXIgbXNvYQ0KICBzaW5nbGVfbXNvYV9kZiA8LSBmaWx0ZXIoZW1wdHlfZGYsIE1TT0ExMUNEID09IG1zb2FOYW1lICYgQ3JpbWUudHlwZT09IkJ1cmdsYXJ5IikNCiAgI2NsZWFuIGRhdGUgZGF0ZQ0KICBzaW5nbGVfbXNvYV9kZiREYXRlU3RyaW5nIDwtIHBhc3RlKHNpbmdsZV9tc29hX2RmJE1vbnRoLCAiLTAxIikNCiAgc2luZ2xlX21zb2FfZGYkRGF0ZUNsZWFuIDwtIHltZChzaW5nbGVfbXNvYV9kZiREYXRlU3RyaW5nKQ0KICAjZ2VuZXJhdGUgdHJhaW5pbmcgc2V0IHVwIHVudGlsIE1hcmNoDQogIHRyYWluaW5nX3NldCA8LSBmaWx0ZXIoc2luZ2xlX21zb2FfZGYsIERhdGVDbGVhbiA8ICIyMDIwLTAzLTAxIikNCiAgI3ByZXBhcmUgZm9yIFByb3BoZXQNCiAgdHJhaW5pbmdfZGYgPC0gdGliYmxlKA0KICAgIGRzPXRyYWluaW5nX3NldCREYXRlQ2xlYW4sDQogICAgeT10cmFpbmluZ19zZXQkY291bnRfYnlfbXNvYSkNCiAgI3N0YXJ0IGFuZCBwcmVkaWN0IHByb3BoZXQgZm9yIDYgbW9udGhzDQogIG0gPC0gcHJvcGhldCh0cmFpbmluZ19kZikNCiAgZnV0dXJlIDwtIG1ha2VfZnV0dXJlX2RhdGFmcmFtZShtLCBwZXJpb2RzID0gNiwgZnJlcSA9ICdtb250aCcpDQogIGZvcmVjYXN0IDwtIHByZWRpY3QobSwgZnV0dXJlKQ0KICBmb3JlY2FzdCRNb250aCA8LSBtb250aChmb3JlY2FzdCRkcykNCiAgZm9yZWNhc3QkWWVhciA8LSB5ZWFyKGZvcmVjYXN0JGRzKQ0KICAjYWdncmVnYXRlIGZvcmVjYXN0cyBhbmQgYWN0dWFsIGNyaW1lDQogIHRoaXNfeWVhciA8LSBmaWx0ZXIoZm9yZWNhc3QsIFllYXIgPiAyMDE5KQ0KICBwZWFrX3BhbmRlbWljIDwtIGZpbHRlcih0aGlzX3llYXIsIE1vbnRoPT0gNCB8IE1vbnRoPT0gNSApDQogIHByZWRpY3Rpb25QaXZvdCA8LSBwZWFrX3BhbmRlbWljICU+JQ0KICAgIGdyb3VwX2J5KE1vbnRoKSAlPiUNCiAgICBzdW1tYXJpemUocHJlZGljdGVkX2J1cmdsYXJ5ID0gbWVhbih5aGF0KSkNCg0KICBzaW5nbGVfbXNvYV9kZiRNb250aE51bSA8LSBtb250aChzaW5nbGVfbXNvYV9kZiREYXRlQ2xlYW4pDQogIHNpbmdsZV9tc29hX2RmJFllYXJOdW0gPC0geWVhcihzaW5nbGVfbXNvYV9kZiREYXRlQ2xlYW4pDQogICNnZW5lcmF0ZSBlcnJvciByYXRlcw0KICB0aGlzX3llYXJfYWN0dWFsIDwtIGZpbHRlcihzaW5nbGVfbXNvYV9kZiwgWWVhck51bSA+IDIwMTkpDQogIHBlYWtfcGFuZGVtaWNfYWN0dWFsIDwtIGZpbHRlcih0aGlzX3llYXJfYWN0dWFsLCBNb250aE51bT09IDQgfCBNb250aE51bT09IDUgKQ0KICBhY3R1YWxfYnVyZ2xhcnkgPC0gc3VtKHBlYWtfcGFuZGVtaWNfYWN0dWFsJGNvdW50X2J5X21zb2EpDQogIHByZWRfYnVyZ2xhcnkgPC0gc3VtKHByZWRpY3Rpb25QaXZvdCRwcmVkaWN0ZWRfYnVyZ2xhcnkpDQogIGVycm9yX2J1cmcgPC0gYWN0dWFsX2J1cmdsYXJ5IC0gcHJlZF9idXJnbGFyeQ0KICBwZXJjZW50YWdlX2Vycm9yX2J1cmcgPC0gZXJyb3JfYnVyZyAvIHByZWRfYnVyZ2xhcnkgDQogIA0KICAjbm93IHJlcGVhdCBmb3Igcm9iYmVyeQ0KICANCiAgc2luZ2xlX21zb2FfZGYgPC0gZmlsdGVyKGVtcHR5X2RmLCBNU09BMTFDRCA9PSBtc29hTmFtZSAmIENyaW1lLnR5cGU9PSJSb2JiZXJ5IikNCiAgc2luZ2xlX21zb2FfZGYkRGF0ZVN0cmluZyA8LSBwYXN0ZShzaW5nbGVfbXNvYV9kZiRNb250aCwgIi0wMSIpDQogIHNpbmdsZV9tc29hX2RmJERhdGVDbGVhbiA8LSB5bWQoc2luZ2xlX21zb2FfZGYkRGF0ZVN0cmluZykNCiAgdHJhaW5pbmdfc2V0IDwtIGZpbHRlcihzaW5nbGVfbXNvYV9kZiwgRGF0ZUNsZWFuIDwgIjIwMjAtMDMtMDEiKQ0KICB0cmFpbmluZ19kZiA8LSB0aWJibGUoDQogICAgZHM9dHJhaW5pbmdfc2V0JERhdGVDbGVhbiwNCiAgICB5PXRyYWluaW5nX3NldCRjb3VudF9ieV9tc29hKQ0KICBtIDwtIHByb3BoZXQodHJhaW5pbmdfZGYpDQogIGZ1dHVyZSA8LSBtYWtlX2Z1dHVyZV9kYXRhZnJhbWUobSwgcGVyaW9kcyA9IDYsIGZyZXEgPSAnbW9udGgnKQ0KICBmb3JlY2FzdCA8LSBwcmVkaWN0KG0sIGZ1dHVyZSkNCiAgZm9yZWNhc3QkTW9udGggPC0gbW9udGgoZm9yZWNhc3QkZHMpDQogIGZvcmVjYXN0JFllYXIgPC0geWVhcihmb3JlY2FzdCRkcykNCiAgdGhpc195ZWFyIDwtIGZpbHRlcihmb3JlY2FzdCwgWWVhciA+IDIwMTkpDQogIHBlYWtfcGFuZGVtaWMgPC0gZmlsdGVyKHRoaXNfeWVhciwgTW9udGg9PSA0IHwgTW9udGg9PSA1ICkNCiAgcHJlZGljdGlvblBpdm90IDwtIHBlYWtfcGFuZGVtaWMgJT4lDQogICAgZ3JvdXBfYnkoTW9udGgpICU+JQ0KICAgIHN1bW1hcml6ZShwcmVkaWN0ZWRfYnVyZ2xhcnkgPSBtZWFuKHloYXQpKQ0KDQogIHNpbmdsZV9tc29hX2RmJE1vbnRoTnVtIDwtIG1vbnRoKHNpbmdsZV9tc29hX2RmJERhdGVDbGVhbikNCiAgc2luZ2xlX21zb2FfZGYkWWVhck51bSA8LSB5ZWFyKHNpbmdsZV9tc29hX2RmJERhdGVDbGVhbikNCg0KICB0aGlzX3llYXJfYWN0dWFsIDwtIGZpbHRlcihzaW5nbGVfbXNvYV9kZiwgWWVhck51bSA+IDIwMTkpDQogIHBlYWtfcGFuZGVtaWNfYWN0dWFsIDwtIGZpbHRlcih0aGlzX3llYXJfYWN0dWFsLCBNb250aE51bT09IDQgfCBNb250aE51bT09IDUgKQ0KICBhY3R1YWxfcm9iYmVyeSA8LSBzdW0ocGVha19wYW5kZW1pY19hY3R1YWwkY291bnRfYnlfbXNvYSkNCiAgcHJlZF9yb2JiZXJ5IDwtIHN1bShwcmVkaWN0aW9uUGl2b3QkcHJlZGljdGVkX2J1cmdsYXJ5KQ0KICBlcnJvcl9yb2IgPC0gYWN0dWFsX3JvYmJlcnkgLSBwcmVkX3JvYmJlcnkNCiAgcGVyY2VudGFnZV9lcnJvcl9yb2IgPC0gZXJyb3Jfcm9iIC8gcHJlZF9yb2JiZXJ5IA0KICANCiAgI2NyZWF0ZSBvdXIgb3V0cHV0IGRhdGFmcmFtZSBhbmQgcmV0dXJuIGl0DQogIA0KICBtc29hX2Vycm9yX3RpYmJsZSA8LSB0aWJibGUoDQogICAgTVNPQTExQ0QgPSBtc29hTmFtZSwgDQogICAgYnVyZ2xhcnlBY3R1YWw9IGFjdHVhbF9idXJnbGFyeSwNCiAgICBidXJnbGFyeVByZWRpY3RlZD0gcHJlZF9idXJnbGFyeSwNCiAgICBidXJnbGFyeUVycm9yPSBlcnJvcl9idXJnLA0KICAgIGJ1cmdsYXJ5UGVyY2VudEVycm9yID0gcGVyY2VudGFnZV9lcnJvcl9idXJnLA0KICAgIHJvYmJlcnlBY3R1YWw9IGFjdHVhbF9yb2JiZXJ5LA0KICAgIHJvYmJlcnlQcmVkaWN0ZWQ9IHByZWRfcm9iYmVyeSwNCiAgICByb2JiZXJ5RXJyb3I9IGVycm9yX3JvYiwNCiAgICByb2JiZXJ5UGVyY2VudEVycm9yPXBlcmNlbnRhZ2VfZXJyb3Jfcm9iDQogICAgKQ0KDQogIHJldHVybihtc29hX2Vycm9yX3RpYmJsZSkNCn0NCg0KZm9yIChtc29hIGluIHVuaXF1ZShlbXB0eV9kZiRNU09BMTFDRCkpew0KICBpdGVyYXRlZF9tc29hX2RmIDwtIGNhbGN1bGF0ZV9lcnJvcihtc29hKQ0KICBtc29hX2Vycm9yX3RpYmJsZSA8LSByYmluZChtc29hX2Vycm9yX3RpYmJsZSwgaXRlcmF0ZWRfbXNvYV9kZikNCiAgDQp9DQoNCg0KYGBgDQoNCg0KDQoNCmBgYHtyfQ0KbXNvYV9lcnJvcl90aWJibGUNCmBgYA0KDQpPdXIgcHJvY2VzcyBoYXMgY29tcGxldGVkOiB3ZSBoYXZlIGEgIkNPVklEIHNoaWZ0IiBtZWFzdXJlIGZvciBhbGwgb2YgTG9uZG9uLg0KDQojIyAzLiBNZWFzdXJpbmcgTG9jYWwgQ09WSUQgQ3JpbWUgU2hpZnRzDQoNCldlIG5vdyBuZWVkIHRvIHVzZSBvdXIgZm9yZWNhc3RzIHRvIG1lYXN1cmUgdGhlICJlcnJvciIgLSB0aGlzIHNob3VsZCBwcm92aWRlIGFuIGluZGljYXRpb24gb2YgdGhlICJDT1ZJRCBDcmltZSBTaGlmdCIsIG9yIGhvdyBtdWNoIHRoZSBhY3R1YWwgY3JpbWUgZGl2ZXJ0ZWQgZnJvbSB0aGUgcHJldmlvdXMgZm9yZWNhc3RzLg0KDQpJIGV4cGxvcmVkIHZhcmlvdXMgYXZlbnVlcyBmb3IgdGhpczogdGhlIGlkZWFsIHNvbHV0aW9uIHdvdWxkIGJlIGEgcmVsYXRpdmUgcmF0ZSBvZiB0aGUgZXJyb3IsIGFzIE1TT0FzIHdpdGggbGFyZ2UgY3JpbWUgbnVtYmVycyB3aWxsIGxpa2VseSBnZW5lcmF0ZSBsYXJnZSBlcnJvcnMsIGFuZCBzbyBhIHJhdGUgd291bGQgYmUgaWRlYWwsIHRob3VnaCB0aGlzIGlzIGNvbXBsaWNhdGVkIGJ5IG91ciBlcnJhdGljIHByZWRpY3Rpb24gYW5kIG1peCBvZiBwb3NpdGl2ZSBhbmQgbmVnYXRpdmUgbnVtYmVycy4NCg0KT3VyIGZpbmFsIHNvbHV0aW9uIGhhcyBleHBsb3JlZCB0d28gb3B0aW9uczoNCi0gdGhlIGFic29sdXRlIGVycm9yIG51bWJlcg0KLSB0aGUgcmVsYXRpdmUgZXJyb3Igb25jZSB0aGUgY3JpbWUgYW5kIHByZWRpY3Rpb25zIGhhdmUgYmVlbiB0cmFuc2Zvcm1lZCAoYnkgYWRkaW5nIDUwKQ0KDQpXZSB2aXN1YWxpc2UgYW5kIGRlc2NyaWJlIHRoZXNlIHN0YXRpc3RpY3MgZmlyc3QgdG8gZW5zdXJlIHRoZXkgYXBwZWFyIHNlbnNpYmxlLg0KDQoNCmBgYHtyfQ0KI3dyaXRlX2Nzdihtc29hX2Vycm9yX3RpYmJsZSwgIm1zb2FfZXJyb3JfdGFibGUyLmNzdiIpDQpgYGANCg0KYGBge3J9DQptc29hX2Vycm9yX3RpYmJsZSA8LSByZWFkX2NzdigibXNvYV9lcnJvcl90YWJsZTIuY3N2IikNCiNtc29hX2Vycm9yX3RpYmJsZTwtIG1zb2FfZXJyb3JfdGFibGVbMjo5ODAsXQ0KbXNvYV9lcnJvcl90aWJibGVbLDI6OV0gPC0gbGFwcGx5KG1zb2FfZXJyb3JfdGliYmxlWywyOjldLCBhcy5udW1lcmljKQ0KDQptc29hX2Vycm9yX3RpYmJsZSA8LSBtc29hX2Vycm9yX3RpYmJsZVsyOjk4MCwgXQ0KDQptc29hX2Vycm9yX3RpYmJsZSA8LSBsZWZ0X2pvaW4obXNvYV9lcnJvcl90aWJibGUsIHJvYmJlcnlfYnlfbXNvYSwgYnkgPSAiTVNPQTExQ0QiKQ0KbXNvYV9lcnJvcl90aWJibGUgPC0gbGVmdF9qb2luKG1zb2FfZXJyb3JfdGliYmxlLCBidXJnbGFyeV9ieV9tc29hLCBieSA9ICJNU09BMTFDRCIpDQoNCg0KYGBgDQoNCg0KYGBge3J9DQoNCm1zb2FfZXJyb3JfdGliYmxlJFJQREJ1cmdsYXJ5IDwtIChtc29hX2Vycm9yX3RpYmJsZSRidXJnbGFyeUFjdHVhbCAtIG1zb2FfZXJyb3JfdGliYmxlJGJ1cmdsYXJ5UHJlZGljdGVkKS8oKG1zb2FfZXJyb3JfdGliYmxlJGJ1cmdsYXJ5UHJlZGljdGVkICsgbXNvYV9lcnJvcl90aWJibGUkYnVyZ2xhcnlBY3R1YWwpLzIpDQoNCm1zb2FfZXJyb3JfdGliYmxlJFJQRFJvYmJlcnkgPC0gKG1zb2FfZXJyb3JfdGliYmxlJHJvYmJlcnlBY3R1YWwgLSBtc29hX2Vycm9yX3RpYmJsZSRyb2JiZXJ5UHJlZGljdGVkKS8oKG1zb2FfZXJyb3JfdGliYmxlJHJvYmJlcnlQcmVkaWN0ZWQgKyBtc29hX2Vycm9yX3RpYmJsZSRyb2JiZXJ5QWN0dWFsKS8yKQ0KDQptc29hX2Vycm9yX3RpYmJsZSRyb2JiZXJ5QWN0dWFsU2hpZnRlZCA8LSBtc29hX2Vycm9yX3RpYmJsZSRyb2JiZXJ5QWN0dWFsICsgNTANCm1zb2FfZXJyb3JfdGliYmxlJHJvYmJlcnlQcmVkaWN0ZWRTaGlmdGVkIDwtIG1zb2FfZXJyb3JfdGliYmxlJHJvYmJlcnlQcmVkaWN0ZWQgKyA1MA0KDQoNCm1zb2FfZXJyb3JfdGliYmxlJFJQRFJvYmJlcnlTaGlmdGVkIDwtIChtc29hX2Vycm9yX3RpYmJsZSRyb2JiZXJ5QWN0dWFsU2hpZnRlZCAtIG1zb2FfZXJyb3JfdGliYmxlJHJvYmJlcnlQcmVkaWN0ZWRTaGlmdGVkKS8oKG1zb2FfZXJyb3JfdGliYmxlJHJvYmJlcnlQcmVkaWN0ZWRTaGlmdGVkICsgbXNvYV9lcnJvcl90aWJibGUkcm9iYmVyeUFjdHVhbFNoaWZ0ZWQpLzIpDQoNCm1zb2FfZXJyb3JfdGliYmxlJGJ1cmdsYXJ5QWN0dWFsU2hpZnRlZCA8LSBtc29hX2Vycm9yX3RpYmJsZSRidXJnbGFyeUFjdHVhbCArIDUwDQptc29hX2Vycm9yX3RpYmJsZSRidXJnbGFyeVByZWRpY3RlZFNoaWZ0ZWQgPC0gbXNvYV9lcnJvcl90aWJibGUkYnVyZ2xhcnlQcmVkaWN0ZWQgKyA1MA0KDQoNCm1zb2FfZXJyb3JfdGliYmxlJFJQRGJ1cmdsYXJ5U2hpZnRlZCA8LSAobXNvYV9lcnJvcl90aWJibGUkYnVyZ2xhcnlBY3R1YWxTaGlmdGVkIC0gbXNvYV9lcnJvcl90aWJibGUkYnVyZ2xhcnlQcmVkaWN0ZWRTaGlmdGVkKS8oKG1zb2FfZXJyb3JfdGliYmxlJGJ1cmdsYXJ5UHJlZGljdGVkU2hpZnRlZCArIG1zb2FfZXJyb3JfdGliYmxlJGJ1cmdsYXJ5QWN0dWFsU2hpZnRlZCkvMikNCg0KDQpgYGANCg0KYGBge3J9DQpwcmludCgiQnVyZ2xhcnkgRXJyb3IiKQ0Kc3VtbWFyeShtc29hX2Vycm9yX3RpYmJsZSRidXJnbGFyeUVycm9yKQ0KcHJpbnQoIkJ1cmdsYXJ5IFJlbGF0aXZlIEVycm9yIikNCnN1bW1hcnkobXNvYV9lcnJvcl90aWJibGUkUlBEYnVyZ2xhcnlTaGlmdGVkKQ0KcHJpbnQoIlJvYmJlcnkgRXJyb3IiKQ0Kc3VtbWFyeShtc29hX2Vycm9yX3RpYmJsZSRyb2JiZXJ5RXJyb3IpDQpwcmludCgiUm9iYmVyeSBSZWxhdGl2ZSBFcnJvciIpDQpzdW1tYXJ5KG1zb2FfZXJyb3JfdGliYmxlJFJQRFJvYmJlcnlTaGlmdGVkKQ0KDQpgYGANCkFzIHdlIGNhbiBzZWUsIHRoZSBhdmVyYWdlIExvbmRvbiBNU09BIGV4cGVyaWVuY2VkIGEgbmVnYXRpdmUgQ09WSUQgY3JpbWUgc2hpZnQgZm9yIGJvdGggYnVyZ2xhcnkgYW5kIHJvYmJlcnksIGJ1dCB0aGlzIGlzIGZhciBmcm9tIGVxdWFsbHkgZGlzdHJpYnV0ZWQgLSBhdCB0aGUgZXh0cmVtZXMsIHNvbWUgYXJlYXMgYWN0dWFsbHkgc2VlIGxhcmdlIGluY3JlYXNlcyBvbiBvdXIgcHJlZGljdGVkIHZhbHVlcy4gDQoNCmBgYHtyICwgZmlnLndpZHRoID0gMTN9DQoNCmJ1cmdfaGlzdCA8LSBnZ3Bsb3QobXNvYV9lcnJvcl90aWJibGUsIGFlcyh4PWJ1cmdsYXJ5RXJyb3IpKSArIGdlb21faGlzdG9ncmFtKCkNCnJvYl9oaXN0IDwtZ2dwbG90KG1zb2FfZXJyb3JfdGliYmxlLCBhZXMoeD1yb2JiZXJ5RXJyb3IpKSArIGdlb21faGlzdG9ncmFtKCkNCmJ1cmdfcl9oaXN0IDwtIGdncGxvdChtc29hX2Vycm9yX3RpYmJsZSwgYWVzKHg9UlBEYnVyZ2xhcnlTaGlmdGVkKSkgKyBnZW9tX2hpc3RvZ3JhbSgpDQpyb2Jfcl9oaXN0IDwtIGdncGxvdChtc29hX2Vycm9yX3RpYmJsZSwgYWVzKHg9UlBEUm9iYmVyeVNoaWZ0ZWQpKSArIGdlb21faGlzdG9ncmFtKCkNCnNjYXR0ZXIgPC0gZ2dwbG90KG1zb2FfZXJyb3JfdGliYmxlLCBhZXMoeCA9IFJQRFJvYmJlcnlTaGlmdGVkLCB5ID0gUlBEYnVyZ2xhcnlTaGlmdGVkKSkgKw0KICBnZW9tX3BvaW50KCkNCg0Kcl9zY2F0dGVyIDwtIGdncGxvdChtc29hX2Vycm9yX3RpYmJsZSwgYWVzKHggPSByb2JiZXJ5RXJyb3IsIHkgPSBidXJnbGFyeUVycm9yKSkgKw0KICBnZW9tX3BvaW50KCkNCg0KZ2dhcnJhbmdlKHJvYl9oaXN0LCBidXJnX2hpc3QsIHJvYl9yX2hpc3QsIGJ1cmdfcl9oaXN0LHNjYXR0ZXIsIHJfc2NhdHRlciwgbmNvbD0yLCBucm93PTMgKQ0KDQoNCmBgYA0KT3VyIHNoaWZ0ZWQgcmVsYXRpdmUgZXJyb3IgcmF0ZSBzZWVtcyB0byBmdW5jdGlvbiBhcyBpbnRlbmRlZDogd2hpbGUgdGhlcmUgYXJlIHN0aWxsIG91dGxpZXJzLCB0aGV5IGFyZSBtb3JlIGNvbmNlbnRyYXRlZCB0aGFuIHRoZXkgYXJlIGZvciB0aGUgcHVyZSBlcnJvciB0ZXJtLCBhbmQgdGhlIG92ZXJhbGwgZGlzdHJpYnV0aW9uIGlzIG1vcmUgZm9jdXNlZCwgd2hpbGUgc3RpbGwgaW5kaWNhdGluZyB0aGUgZGlyZWN0aW9uIGFuZCByZWxhdGl2ZSBzdHJlbmd0aCBvZiBvdXIgQ09WSUQgZWZmZWN0Lg0KDQpMZXQncyBtYXAgdGhpcyBlZmZlY3QgdmlzdWFsbHksIGFuZCBzZWUgaWYgYW55IHBhcnRpY3VsYXIgYXJlYXMgc3RhbmQgb3V0Lg0KDQpgYGB7ciAsIGZpZy53aWR0aCA9IDEzfQ0KI3JlLWluZ2VzdCBvdXIgZ2VvZ3JhcGhpYyBNU09BIGJvcmRlcnMNCm1zb2FfYm9yZGVycyA8LSBzdF9yZWFkKCJtc29hX2JvcmRlcnMvTVNPQV8yMDExX0xvbmRvbl9nZW5fTUhXLnRhYiIsIGNycz0yNzcwMCkNCg0KZ2VvZ3JhcGhpY19lcnJvcl9tYXAgPC0gbGVmdF9qb2luKG1zb2FfYm9yZGVycywgbXNvYV9lcnJvcl90aWJibGUsIGJ5ID0gIk1TT0ExMUNEIikNCg0KYnVyZ19tYXAgPC0gdG1fc2hhcGUoZ2VvZ3JhcGhpY19lcnJvcl9tYXApICsNCiAgdG1fZmlsbChjb2wgPSAicm9iYmVyeUVycm9yIiwgdGl0bGUgPSAiUm9iYmVyeSBFcnJvciIsIHBhbGV0dGU9Ii1QdU9yIikrDQogIHRtX2xheW91dChsZWdlbmQub3V0c2lkZSA9IFRSVUUsIGxlZ2VuZC5vdXRzaWRlLnBvc2l0aW9uID0gInJpZ2h0IikNCnJvYl9tYXAgPC10bV9zaGFwZShnZW9ncmFwaGljX2Vycm9yX21hcCkgKw0KICB0bV9maWxsKGNvbCA9ICJidXJnbGFyeUVycm9yIiwgdGl0bGUgPSAiQnVyZ2xhcnkgIEVycm9yIiwgcGFsZXR0ZT0iLVB1T3IiKSsNCiAgdG1fbGF5b3V0KGxlZ2VuZC5vdXRzaWRlID0gVFJVRSwgbGVnZW5kLm91dHNpZGUucG9zaXRpb24gPSAicmlnaHQiKQ0KDQoNCmJ1cmdfbWFwX3JhdGUgPC0gdG1fc2hhcGUoZ2VvZ3JhcGhpY19lcnJvcl9tYXApICsNCiAgdG1fZmlsbChjb2wgPSAiUlBEUm9iYmVyeVNoaWZ0ZWQiLCB0aXRsZSA9ICJSb2JiZXJ5IEVycm9yIFJlbGF0aXZlIiwgcGFsZXR0ZT0iLVB1T3IiKSsNCiAgdG1fbGF5b3V0KGxlZ2VuZC5vdXRzaWRlID0gVFJVRSwgbGVnZW5kLm91dHNpZGUucG9zaXRpb24gPSAicmlnaHQiKQ0Kcm9iX21hcF9yYXRlIDwtdG1fc2hhcGUoZ2VvZ3JhcGhpY19lcnJvcl9tYXApICsNCiAgdG1fZmlsbChjb2wgPSAiUlBEYnVyZ2xhcnlTaGlmdGVkIiwgdGl0bGUgPSAiQnVyZ2xhcnkgIEVycm9yIFJlbGF0aXZlIiwgcGFsZXR0ZT0iLVB1T3IiKSsNCiAgdG1fbGF5b3V0KGxlZ2VuZC5vdXRzaWRlID0gVFJVRSwgbGVnZW5kLm91dHNpZGUucG9zaXRpb24gPSAicmlnaHQiKQ0KDQoNCnRtYXBfYXJyYW5nZShidXJnX21hcCwgcm9iX21hcCwgYnVyZ19tYXBfcmF0ZSwgcm9iX21hcF9yYXRlICwgbnJvdyA9IDIsIG5jb2w9MikNCg0KYGBgDQpJdCdzIGhhcmQgdG8gaWRlbnRpZnkgYW55IG9idmlvdXMgZWZmZWN0IHZpc3VhbGx5LCBidXQgd2UgZG8gbm90aWNlIHRoYXQgd2hpbGUgY2VudHJhbCBMb25kb24gc2VlcyBzb21lIHZlcnkgc3Ryb25nIHJlZHVjdGlvbnMsIGl0IGFsc28gc2VlcyBzb21lIGluY3JlYXNlcy4gIENvbnZlcnNlbHksIHRoZSBvdXRza2lydHMgb2YgTG9uZG9uIChub3RhYmx5IHRvIHRoZSBzb3V0aCBhbmQgV2VzdCkgYXJlIGEgbmVhciBjb250aW51b3VzIGFyZWEgb2YgbGFyZ2UgZGVjcmVhc2VzLiBUaGUgZWZmZWN0IGRvZXMgdmFyeSBieSBvZmZlbmNlIHR5cGUsIGJ1dCB0aGUgcGF0dGVybiBzZWVuIGluIFNvdXRoIGFuZCBXZXN0IExvbmRvbiBhcHBlYXJzIGJyb2FkbHkgY29uc2lzdGVudC4NCg0KIyMgNC4gSWRlbnRpZnlpbmcgQ29ycmVsYXRlcyBhbmQgTW9kZWxsaW5nDQoNCldlJ3ZlIGlkZW50aWZpZWQgdGhhdCB0aGUgQ09WSUQgY3JpbWUgZWZmZWN0IHdhcyBmZWx0IHVuZXF1YWxseSBhY2Nyb3NzIExvbmRvbiwgYW5kIHZhcmllcyBieSBvZmZlbmNlIHR5cGUuIFRvIGZpbmFsaXNlIG91ciBwcm9qZWN0LCB3ZSB3aWxsIGJlIGxpbmtpbmcgb3VyIGRhdGEgdG8gW2RlbW9ncmFwaGljIGRhdGEgcHJvdmlkZWQgYnkgTU9QQUNdKGh0dHBzOi8vZGF0YS5sb25kb24uZ292LnVrL2RhdGFzZXQvbXNvYS1hdGxhcyksIGFuZCBhaW1pbmcgdG8gdXNlIGl0IHRvIGlkZW50aWZ5IGNvcnJlbGF0ZXMgdG8gb3VyICJjb3ZpZCBzaGlmdCIsIGFuZCBob3BlZnVsbHkgYnVpbGQgbW9kZWxzIGRpc2VudGFuZ2xpbmcgdGhlIGVmZmVjdC4NCg0KYGBge3J9DQpsaWJyYXJ5KHJlYWR4bCkNCiNpbmdlc3QgQVRMQVMNCm1zb2FfYXRsYXMgPC0gcmVhZF9leGNlbCgibXNvYV9hdGxhcy9tc29hLWRhdGEueGxzIikNCg0KI2pvaW4gYnkgTVNPQQ0KZ2VvZ3JhcGhpY19tc29hX21hdHJpeCA8LSBsZWZ0X2pvaW4oZ2VvZ3JhcGhpY19lcnJvcl9tYXAsIG1zb2FfYXRsYXMsIGJ5ID0gIk1TT0ExMUNEIikNCg0KI2NvbnZlcnQgdG8gdGliYmxlDQptc29hX21hdHJpeF90YmwgPC0gYXNfdGliYmxlKGdlb2dyYXBoaWNfbXNvYV9tYXRyaXgpDQp3cml0ZV9jc3YobXNvYV9tYXRyaXhfdGJsLCAibXNvYV9tYXRyaXguY3N2IikNCg0KDQojc2VsZWN0IG9ubHkgbnVtZXJpYyBkYXRhDQptc29hX21hdHJpeF9udW1lcmljIDwtZHBseXI6OnNlbGVjdF9pZihtc29hX21hdHJpeF90YmwsIGlzLm51bWVyaWMpDQptc29hX21hdHJpeF9udW1lcmljDQpgYGANCkhhdmluZyBub3cgaW5nZXN0ZWQgYW5kIGxpbmtlZCBvdXIgZGF0YSwgd2UgYmVnaW4gYnkgZXhwbG9yaW5nIHRoZSBmYWN0b3JzIG1vc3QgaGlnaGx5IGNvcnJlbGF0ZWQgd2l0aCBvdXIgcmVsYXRpdmUgZXJyb3IgcmF0ZXMuDQoNCmBgYHtyLCByZXN1bHRzPSdoaWRlJ30NCg0KY29ycl9kZiA8LSBjb3JyZWxhdGUoZHBseXI6OnNlbGVjdF9pZihtc29hX21hdHJpeF90YmwsIGlzLm51bWVyaWMpLCBxdWlldCA9IFRSVUUpDQoNCm9wdGlvbnMoc2NpcGVuPTk5OSkNCmBgYA0KU3RhcnRpbmcgYnkgb3VyIHJlbGF0aXZlIHJvYmJlcnkgZXJyb3IsIGEgZmV3IGludGVyZXN0aW5nIGNvcnJlbGF0ZXMgc3RhbmQgb3V0Og0KLSByb2FkIHRyYWZmaWMgY2FzdWFsdGllcw0KLSBidXJnbGFyeSBudW1iZXJzDQotIHRoZSBudW1iZXIgb2YgZHdlbGxpbmdzIHdpdGggbm8gdXN1YWwgcmVzaWRlbnRzLCBhbmQgdGhlIG51bWJlciBvZiBjb21tZXJjaWFsIHJlc2lkZW50cw0KLSBob3VzZWhvbGRzIHdpdGggbm8gY2Fycw0KLSB0aGUgYWdlIGNvbXBvc2l0aW9uIG9mIHRoZSBhcmVhDQotIGdlbmVyYWwgZGVwcml2YXRpb24gaW5kaWNhdG9ycyAoc3VjaCBhcyB0aGUgcHJvcG9ydGlvbiBvZiBob3VzZWhvbGRzIHdpdGggY2VudHJhbCBoZWF0aW5nKQ0KDQpgYGB7cn0NCiNzaG93IG9ubHkgY29ycmVsYXRlcyB3aXRoIGFuIGFic29sdXRlIHZhbHVlIGhpZ2hlciB0aGFuIDAuMg0KI2ZpbHRlcihkcGx5cjo6c2VsZWN0KGNvcnJfZGZbb3JkZXIoY29ycl9kZiRSUERSb2JiZXJ5U2hpZnRlZCksXSAsIHRlcm0sIFJQRFJvYmJlcnlTaGlmdGVkKSwgUlBEUm9iYmVyeVNoaWZ0ZWQgPCAtMC4yIHwgUlBEUm9iYmVyeVNoaWZ0ZWQgPiAwLjIpDQojZmlsdGVyKGRwbHlyOjpzZWxlY3QoY29ycl9kZltvcmRlcihjb3JyX2RmJFJQRFJvYmJlcnlTaGlmdGVkKSxdICwgdGVybSwgUlBEUm9iYmVyeVNoaWZ0ZWQpLCBSUERSb2JiZXJ5U2hpZnRlZCA8IC0wLjIgfCBSUERSb2JiZXJ5U2hpZnRlZCA+IDAuMikNCmhpZ2hfY29ycl9yb2IgPC0gZmlsdGVyKGRwbHlyOjpzZWxlY3QoY29ycl9kZiwgdGVybSwgUlBEUm9iYmVyeVNoaWZ0ZWQpLCBSUERSb2JiZXJ5U2hpZnRlZCA8IC0wLjE1IHwgUlBEUm9iYmVyeVNoaWZ0ZWQgPiAwLjE1KQ0KDQpoaWdoX2NvcnJfcm9iW29yZGVyKGhpZ2hfY29ycl9yb2IkUlBEUm9iYmVyeVNoaWZ0ZWQpLF0NCg0KYGBgDQpUaGUgY29ycmVsYXRpb25zIGZvciBidXJnbGFyeSBhcmUgd2Vha2VyIC0gb25seSBhIGZldyBoYXZlIGFuIGFic29sdXRlIHZhbHVlIGhpZ2hlciB0aGFuIDAuMiAtIGJ1dCBhIGZldyBzdGFuZCBvdXQ6DQotIGhvdXNlaG9sZHMgd2l0aCBubyByZXNpZGVudHMNCi0gaGlnaCByb2JiZXJ5IG51bWJlcnMNCi0gaG91c2UgcHJpY2VzIA0KDQpgYGB7cn0NCg0KaGlnaF9jb3JyX2J1cmcgPC0gZmlsdGVyKGRwbHlyOjpzZWxlY3QoY29ycl9kZiwgdGVybSwgUlBEYnVyZ2xhcnlTaGlmdGVkKSwgUlBEYnVyZ2xhcnlTaGlmdGVkIDwgLTAuMTUgfCBSUERidXJnbGFyeVNoaWZ0ZWQgPiAwLjE1KQ0KDQpoaWdoX2NvcnJfYnVyZ1tvcmRlcihoaWdoX2NvcnJfYnVyZyRSUERidXJnbGFyeVNoaWZ0ZWQpLF0NCg0KYGBgDQpUaGVzZSBjb3JyZWxhdGVzIHN1Z2dlc3Qgd2UgY2FuIG1vZGVsIHRoaXMgc2hpZnQgLSB0aGlzIGlzIGxpa2VseSB0byBwcm92ZSBtb3JlIHJlbGlhYmxlIGZvciByb2JiZXJ5ICh3aGVyZSB0aGUgY29ycmVsYXRpb25zIGFyZSBzdHJvbmdlciksIGFuZCBzZWVtIGxpbmtlZCB0byB1c3VhbCByZXNpZGVudCBwb3B1bGF0aW9uKGFzIG1lYXN1cmVkIGJ5IGhvdXNlaG9sZCBjb21wb3NpdGlvbiksIGRlcHJpdmF0aW9uICh0aHJvdWdoIHZhcmlvdXMgcHJveHkgaW5kaWNhdG9ycyBzdWNoIGFzIGNlbnRyYWwgaGVhdGluZyBwcmVzZW5jZSBvciBob3VzaW5nIHR5cGUpLCBhbmQgZ2VuZXJhbCBjcmltZSBwYXR0ZXJucyAodGhyb3VnaCB0b3RhbCBidXJnbGFyeSBhbmQgcm9iYmVyeSBudW1iZXJzKQ0KDQpXZSB3aWxsIHRha2UgdHdvIGFwcHJvYWNoZXMgZm9yIG1vZGVsbGluZzogYSBzaW1wbGUgcmVncmVzc2lvbiAodG8gaWRlbnRpZnkgc3Ryb25nIGxpbmtzKSBhbmQgcmFuZG9tIGZvcmVzdCByZWdyZXNzb3JzICh0byBpZGVudGlmeSBub24tbGluZWFyIGFzc29jaWF0aW9ucykNCg0KIyMjIFNpbXBsZSBSZWdyZXNzaW9uDQoNCldlIGJlZ2luIHRocm91Z2ggdGhlIHVzZSBvZiBzaW1wbGUgT0xTIHJlZ3Jlc3Npb24uIFRoaXMgaXMgYSBsaW5lYXIgbW9kZWwgdGhhdCBoYXMgbGFyZ2UgbGltaXRhdGlvbnMgZm9yIG1vZGVsbGluZyBjb21wbGV4IHJlbGF0aW9uc2hpcHMsIGJ1dCBjYW4gYmUgYW4gZWZmZWN0aXZlIGZpcnN0IHN0ZXAsIGVmZmVjdGl2ZWx5IHdpdGggYSBmZXcgdHJhbnNmb3JtYXRpb25zLg0KDQpSIGRvZXMgbm90IGNvcGUgd2VsbCB3aXRoIGJsYW5rIHNwYWNlcyBpbiB0ZXJtcywgc28gd2UnbGwgZXh0cmFjdCBhbmQgcmVuYW1lIG91ciBrZXkgY29ycmVsYXRlcy4NCg0KYGBge3J9DQoNCiNtYWtlIGNvcHkgb2YgZGYgYW5kIHJlbmFtZSANCg0KbXNvYV9jb3B5IDwtIG1zb2FfbWF0cml4X251bWVyaWMNCg0KbmFtZXMobXNvYV9jb3B5KVtuYW1lcyhtc29hX2NvcHkpID09ICJEd2VsbGluZyB0eXBlICgyMDExKSBIb3VzZWhvbGQgc3BhY2VzIHdpdGggbm8gdXN1YWwgcmVzaWRlbnRzIl0gPC0gIkR3ZWxsaW5nTm9SZXNpZGVudHMiDQpuYW1lcyhtc29hX2NvcHkpW25hbWVzKG1zb2FfY29weSkgPT0gIkhvdXNlIFByaWNlcyBNZWRpYW4gSG91c2UgUHJpY2UgKMKjKSAyMDEwIl0gPC0gIk1lZGlhbkhvdXNlUHJpY2UiDQpuYW1lcyhtc29hX2NvcHkpW25hbWVzKG1zb2FfY29weSkgPT0gIkR3ZWxsaW5nIHR5cGUgKDIwMTEpIEZsYXQsIG1haXNvbmV0dGUgb3IgYXBhcnRtZW50Il0gPC0gIkZsYXRBcHJ0Ig0KbmFtZXMobXNvYV9jb3B5KVtuYW1lcyhtc29hX2NvcHkpID09ICJRdWFsaWZpY2F0aW9ucyAoMjAxMSBDZW5zdXMpIFNjaG9vbGNoaWxkcmVuIGFuZCBmdWxsLXRpbWUgc3R1ZGVudHM6IEFnZSAxOCBhbmQgb3ZlciJdIDwtICJmdWxsVGltZVN0dWRlbnRzIg0KbmFtZXMobXNvYV9jb3B5KVtuYW1lcyhtc29hX2NvcHkpID09ICJDYXIgb3IgdmFuIGF2YWlsYWJpbGl0eSAoMjAxMSBDZW5zdXMpIE5vIGNhcnMgb3IgdmFucyBpbiBob3VzZWhvbGQiXSA8LSAiTm9DYXJzIg0KbmFtZXMobXNvYV9jb3B5KVtuYW1lcyhtc29hX2NvcHkpID09ICJFdGhuaWMgR3JvdXAgKDIwMTEgQ2Vuc3VzKSBPdGhlciBldGhuaWMgZ3JvdXAiXSA8LSAiT3RoZXJFdGhuaWNHcm91cCINCm5hbWVzKG1zb2FfY29weSlbbmFtZXMobXNvYV9jb3B5KSA9PSAiQ2VudHJhbCBIZWF0aW5nICgyMDExIENlbnN1cykgSG91c2Vob2xkcyB3aXRoIGNlbnRyYWwgaGVhdGluZyAoJSkiXSA8LSAiQ2VudHJhbEhlYWxpbmdQZXJjZW50Ig0KDQoNCg0KYGBgDQpMZXQncyBub3cgZXh0cmFjdCB0aGVzZSB0byBhIHNlcGFyYXRlIGRhdGFmcmFtZSwgcmVtb3ZlIGFueSBtaXNzaW5nIHZhbHVlcywgYW5kIHByb3ZpZGUgc29tZSBxdWljayBzdW1tYXJ5IHN0YXRpc3RpY3MgdG8gaWRlbnRpZnkgYW55IG9idmlvdXMgY29uY2VybnMuDQoNCmBgYHtyfQ0KDQpmZWF0dXJlX2RmIDwtIGRwbHlyOjpzZWxlY3QobXNvYV9jb3B5LCBSUERidXJnbGFyeVNoaWZ0ZWQsIFJQRFJvYmJlcnlTaGlmdGVkLCB0b3RhbF9idXJnbGFyaWVzLCB0b3RhbF9yb2JiZXJpZXMsIER3ZWxsaW5nTm9SZXNpZGVudHMsIE1lZGlhbkhvdXNlUHJpY2UsIEZsYXRBcHJ0LCBmdWxsVGltZVN0dWRlbnRzLCBOb0NhcnMsIE90aGVyRXRobmljR3JvdXAsIENlbnRyYWxIZWFsaW5nUGVyY2VudCwgQXZIaG9sZFN6LCBDb21Fc3RSZXMpDQoNCmZlYXR1cmVfZGYgPC0gZHJvcF9uYShmZWF0dXJlX2RmLCBSUERidXJnbGFyeVNoaWZ0ZWQsIFJQRFJvYmJlcnlTaGlmdGVkKQ0KZmVhdHVyZV9kZg0KYGBgDQpgYGB7cn0NCnN1bW1hcnkoZmVhdHVyZV9kZikNCmBgYA0KDQoNCmBgYHtyfQ0KDQpjb2xTdW1zKGlzLm5hKGZlYXR1cmVfZGYpKQ0KYGBgDQpBcyBhIHF1aWNrIGluaXRpYWwgZXhlcmNpc2UsIHdlJ2xsIGNyZWF0ZSAgYSBzY2F0dGVyIHBsb3Qgb2YgdGhlIGludGVyYWN0aW9uIGJldHdlZW4gZWFjaCBhbmQgZXZlcnkgb25lIG9mIG91ciB2YXJpYWJsZXMuIFRoaXMgd2lsbCBwcm9iYWJseSBiZSB0b28gbWVzc3kgdG8gbWFrZSBhbnkgcmVhbCBmaW5kaW5ncywgYnV0IGNhbiBzZXJ2ZSB0byBxdWlja2x5IGhpZ2hsaWdodCBzdHJvbmcgYXNzb2NpYXRpb25zLg0KDQpgYGB7ciwgZmlnLndpZHRoID0gMTN9DQpwYWlycyhmZWF0dXJlX2RmKQ0KDQpgYGANCkFzIHdlIGhvcGVkLCBzb21lIG9idmlvdXMgcmVsYXRpb25zaGlwcyBzdGFuZCBvdXQ6IGZvciBpbnN0YW5jZSwgdGhlIHByZXNlbmNlIG9mIGFwYXJ0bWVudHMsIGFuZCBob3VzZWhvbGRzIHdpdGggbm8gY2Fycywgb3IgY2VudHJhbCBoZWF0aW5nIGFuZCBhdmVyYWdlIGhvdXNlaG9sZCBzaXplLg0KDQpPdXIgcm9iYmVyeSBhbmQgYnVyZ2xhcnkgZGF0YSBhbmQgY2hhbmdlIHJhdGVzIGFyZSBkZW5zZWx5IGNsdXN0ZXJlZCAtIHRoZXkncmUgdW5saWtlbHkgdG8gY2xlYW5seSBhc3NvY2lhdGUgd2l0aCBhbnl0aGluZy4gV2l0aCB0aGF0IGluIG1pbmQsIHdlJ2xsIHBlcmZvcm0gYSBsb2cgdHJhbnNmb3JtYXRpb24uIFRoaXMgY2Fubm90IGJlIHVuZGVydGFrZW4gd2l0aCBuZWdhdGl2ZSB2YWx1ZXMsIHNvIG9uY2UgYWdhaW4gd2UnbGwgcGVyZm9ybSBhIHNoaWZ0IChvZiAyKSBmb3IgYm90aCBvZiBvdXIgcmVsYXRpdmUgZXJyb3IgbnVtYmVycywgYXMgd2VsbCBhcyBhIGNvbW1lcmNpYWwgcmVzaWRlbnQgY29sdW1uLCBiZWZvcmUgbG9nIHRyYW5zZm9ybWluZyBvdXIgZmVhdHVyZXMuDQoNCg0KDQoNCmBgYHtyfQ0KZmVhdHVyZV9kZiRCdXJnbGFyeVJQRFRyYW5mb3JtIDwtIGZlYXR1cmVfZGYkUlBEYnVyZ2xhcnlTaGlmdGVkICAgICsgMg0KZmVhdHVyZV9kZiRSb2JiZXJ5UlBEVHJhbmZvcm0gPC0gZmVhdHVyZV9kZiRSUERSb2JiZXJ5U2hpZnRlZCAgICsgMg0KDQpmZWF0dXJlX2RmJENvbUVzdFJlc1RyYW5mb3JtIDwtIGZlYXR1cmVfZGYkQ29tRXN0UmVzICAgKyAyDQoNCg0KZm9yIChjb2wgaW4gY29sbmFtZXMoZmVhdHVyZV9kZikpew0KICBuZXdfbmFtZSA8LSBwYXN0ZSgibG9nXyIsIGNvbCwgc2VwID0gIiIpDQogIGZlYXR1cmVfZGZbbmV3X25hbWVdIDwtIGxvZyhmZWF0dXJlX2RmW2NvbF0pDQp9DQpgYGANCmBgYHtyfQ0KDQpkcm9wPC0gYyggImxvZ19Db21Fc3RSZXMiLCAibG9nX1JQRGJ1cmdsYXJ5U2hpZnRlZCIsICJsb2dfUlBEUm9iYmVyeVNoaWZ0ZWQiKQ0KZmVhdHVyZV9kZjwtIGZlYXR1cmVfZGZbLCEobmFtZXMoZmVhdHVyZV9kZikgJWluJSBkcm9wKV0NCg0KDQoNCmBgYA0KDQoNCg0KYGBge3J9DQoNCmZlYXRfdHJhbnNmb3JtX2RmIDwtIGZlYXR1cmVfZGZbLDE3OjI5XQ0Kb3JpZ19mZWF0X2RmIDwtIGZlYXR1cmVfZGZbLDA6MTddDQoNCg0KYGBgDQpXZSd2ZSBub3cgc2VwYXJhdGVkIGEgc2VwYXJhdGUgZGF0YWZyYW1lIHdoZXJlIGVhY2ggdmFsdWUgaGFzIGJlZW4gbG9nIHRyYW5zZm9ybWVkIC0gd2hpbGUgdGhpcyBpc24ndCBodWdlbHkgcmlnb3JvdXMgKGFuZCB3b3VsZCBiZW5lZml0IGZyb20gaW5zcGVjdGluZyB0aGUgcmVsYXRpb25zaGlwcyBpbiBtb3JlIGRldGFpbCkgaXQgc2VydmVzIG91ciBpbW1lZGlhdGUgcHVycG9zZS4NCg0KYGBge3IsIGZpZy53aWR0aCA9IDEzfQ0KcGFpcnMoZmVhdF90cmFuc2Zvcm1fZGYpDQoNCmBgYA0KDQpXaGlsZSB3ZSd2ZSBpbnRyb2R1Y2VkIGEgYml0IG9mIG5vaXNlLCB3ZSd2ZSBhbHNvICJmb3JjZWQiIHNvbWUgb2Ygb3VyIHZhcmlhYmxlcyBpbnRvIHJlbGF0aW9uc2hpcHMgdGhhdCBsb29rIHNlbWkgbGluZWFyLiANCg0KVG8gZGlnIGludG8gdGhpcyBkZWVwZXIsIGxldCdzIGNyZWF0ZSBhIGNvcnJlbGF0aW9uIG1hdHJpeCBmb3Igb3VyIGVudGlyZSB0cmFuc2Zvcm1lZCBkYXRhZnJhbWUuDQoNCmBgYHtyfQ0KY29ycmVsYXRlKGZlYXRfdHJhbnNmb3JtX2RmKQ0KDQoNCmBgYA0KDQpUbyBwcm92aWRlIGEgdmlzdWFsIGFpZCwgSSd2ZSBleHRyYWN0ZWQgdGhlIGNvbHVtbiBmb3Igb3VyIHJvYmJlcnkgcmVsYXRpdmUgY2hhbmdlIHJhdGUsIGFuZCBzb3J0ZWQgdGhlIHRhYmxlIGFjY29yZGluZ2x5Lg0KDQoNCmBgYHtyfQ0KZHBseXI6OnNlbGVjdChjb3JyZWxhdGUoZmVhdF90cmFuc2Zvcm1fZGYpW29yZGVyKGNvcnJlbGF0ZShmZWF0X3RyYW5zZm9ybV9kZikkbG9nX0J1cmdsYXJ5UlBEVHJhbmZvcm0pLF0sIHRlcm0sIGxvZ19CdXJnbGFyeVJQRFRyYW5mb3JtKQ0KYGBgDQojIyMgUmVncmVzc2lvbg0KTm93IHRoYXQgd2UndmUgdHJhbnNmb3JtZWQgb3VyIGRhdGEsIGNsZWFuZWQgaXQgdXAsIGFuZCBpZGVudGlmaWVkIHBvdGVudGlhbCBjb3JyZWxhdGVzLCBsZXQncyBidWlsZCBvdXIgbGluZWFyIG1vZGVsLiANCg0KVGhlcmUgYXJlIHZhcmlvdXMgYXV0b21hdGVkIHRvb2xzIGZvciB0aGlzIHByb2Nlc3MgdGhhdCBzZWVrIHRvIHByb3ZpZGUgdGhlIGhpZ2hlc3QgZml0IGFuZCBzaWduaWZpY2FuY2UsIGJ1dCBnaXZlbiB0aGUgaGlnaCBkZWdyZWUgb2YgY29ycmVsYXRpb24gYmV0d2VlbiBteSBjaG9zZW4gZmVhdHVyZXMsIEkndmUgdGFrZW4gYSBtb3JlIG1hbnVhbCBhcHByb2FjaCBhbmQgdGVzdGVkIGEgdmFyaWV0eSBvZiBtb2RlbHMgdW50aWwgSSBpZGVudGlmaWVkIG9uZSB3aXRoIGEgc3VpdGFibGUgZml0LiAgVGhlIGZpbmFsIG1vZGVsIGlzIGJlbG93Lg0KDQpgYGB7cn0NCm1vZF9idXJnbGFyeSA8LSBsbShsb2dfQnVyZ2xhcnlSUERUcmFuZm9ybSB+IGxvZ190b3RhbF9idXJnbGFyaWVzICsgbG9nX0ZsYXRBcHJ0ICsgbG9nX01lZGlhbkhvdXNlUHJpY2UgICsgbG9nX0NvbUVzdFJlc1RyYW5mb3JtLCBkYXRhID0gZmVhdF90cmFuc2Zvcm1fZGYpDQpzdW1tYXJ5KG1vZF9idXJnbGFyeSkNCmBgYA0KDQpPdXIgbW9kZWwgc3VnZ2VzdHMgdGhlIGxhcmdlc3QgYnVyZ2xhcnkgZGVjcmVhc2UgbGlua2VkIHRvIGxvY2tkb3duIHdhcyBpbiBNU09BcyB3aGljaCBhIGhpZ2ggbGV2ZWwgb2YgaGlzdG9yaWMgYnVyZ2xhcnkuIFRoZSBjb21wb3NpdGlvbiBvZiBob3VzaW5nL2FjY29tb2RhdGlvbiBhbmQgYXJlYSB0eXBlIGFsc28gc2VlbXMgdG8gcGxheSBhIHJvbGUsIHdpdGggdGhvc2UgYXJlYXMgd2l0aCBoaWdoZXIgbWVkaWFuIGhvdXNlIHByaWNlcywgYW5kIGEgbGFyZ2VyIG51bWJlciBvZiBjb21tZXJjaWFsIHJlc2lkZW50cywgc2VlaW5nIHN0cm9uZ2VyIGRlY3JlYXNlcywgd2hpbGUgY29udmVyc2VseSBhcmVhcyB3aXRoIGxhcmdlIG51bWJlciBvZiBhcGFydG1lbnRzIHRlbXBlciB0aGUgZWZmZWN0Lg0KDQpXaGlsZSBhbGwgb3VyIHZhcmlhYmxlcyBhcmUgc2lnbmlmaWNhbnQsIHRoZSBtb2RlbCBpcyBub3QgYSBwYXJ0aWN1bGFybHkgZ29vZCBmaXQgLSB0aGUgYWRqdXN0ZWQgUjIgaXMgYXJvdW5kIDAuMDgsIHN1Z2dlc3RpbmcgdGhhdCBsZXNzIHRoYW4gMTAlIG9mIHRoZSB2YXJpYW5jZSBpcyBhY2NvdW50ZWQgZm9yIGJ5IG91ciBtb2RlbC4gSSBzdXNwZWN0IG1vcmUgZ2VvZ3JhcGhpYyBmZWF0dXJlcyAtIHN1Y2ggYXMgZGlzdGFuY2UgZnJvbSBjZW50cmFsIExvbmRvbiwgbW9yZSBhY2N1cmF0ZSBmb290ZmFsbCwgb3Igc3BhdGlhbCBsYWdzIC0gd291bGQgcHJvYmFibHkgYmUgdXNlZnVsLCBidXQgdGhhdCdzIG91dHNpZGUgdGhlIHNjb3BlIG9mIHRoaXMgcHJvamVjdC4NCg0KTGV0J3MgcGVyZm9ybSBhIHNpbWlsYXIgZXhlcmNpc2UgZm9yIHJvYmJlcnkuDQoNCmBgYHtyfQ0KZHBseXI6OnNlbGVjdChjb3JyZWxhdGUoZmVhdF90cmFuc2Zvcm1fZGYpW29yZGVyKGNvcnJlbGF0ZShmZWF0X3RyYW5zZm9ybV9kZikkbG9nX1JvYmJlcnlSUERUcmFuZm9ybSksXSwgdGVybSwgbG9nX1JvYmJlcnlSUERUcmFuZm9ybSkNCmBgYA0KDQoNCmBgYHtyfQ0KbW9kX2J1cmdsYXJ5IDwtIGxtKGxvZ19Sb2JiZXJ5UlBEVHJhbmZvcm0gfiBsb2dfdG90YWxfcm9iYmVyaWVzICArIGxvZ190b3RhbF9idXJnbGFyaWVzICsgbG9nX0ZsYXRBcHJ0ICsgbG9nX0NlbnRyYWxIZWFsaW5nUGVyY2VudCArIGxvZ19mdWxsVGltZVN0dWRlbnRzLCBkYXRhID0gZmVhdF90cmFuc2Zvcm1fZGYpDQpzdW1tYXJ5KG1vZF9idXJnbGFyeSkNCmBgYA0KDQpUaGlzIGlzIGEgbm90YWJseSBiZXR0ZXIgZml0IHRoYW4gb3VyIGJ1cmdsYXJ5IG1vZGVsLCB3aXRoIG91ciBSMiBzdWdnZXN0aW5nIHdlIG5vdyBhY2NvdW50IGZvciBvdmVyIDIwJSBvZiB0aGUgdmFyaWFuY2UuIFRoZSBuYXR1cmUgb2Ygb3VyIHByZWRpY3RvcnMgaXMgYWxzbyBxdWl0ZSBkaWZmZXJlbnQ6IHdoaWxlIHdlIHN0aWxsIHNlZSBhIG5lZ2F0aXZlIHJlbGF0aW9uc2hpcCB3aXRoIGhpc3RvcmljIGNyaW1lICh3aXRoIGFyZWFzIG9mIGhpZ2ggaGlzdG9yaWMgY3JpbWUgZXhwZXJpZW5jaW5nIGxhcmdlciByZWxhdGl2ZSBkZWNyZWFzZXMpLCB0aGVyZSBpcyBhIHBvc2l0aXZlIHJlbGF0aW9uc2hpcCB3aXRoIGJvdGggdGhlIHByZXNlbmNlIG9mIGFwYXJ0bWVudHMgYW5kIGNlbnRyYWwgaGVhdGluZy4NCg0KSSBzdXNwZWN0IHNvbWUgb2YgdGhlc2UgZmVhdHVyZXMgYXJlIGNvcnJlbGF0ZXMgb2YgZGVwcml2YXRpb24sIHNvIEkgd2FudCB0byBjcmVhdGUgYSBxdWljayBzY2F0dGVyIG9mIHRocmVlIC0gZm9yIG5vdyB3ZSdsbCBkbyBpdCBhZ2FpbnN0IG1lZGlhbiBob3VzZSBwcmljZSwgd2hpY2ggaXMgZGVmaW5pdGVseSBkZXByaXZhdGlvbiBjb3JyZWxhdGVkLg0KDQpgYGB7cn0NCg0KaGVhdGluZyA8LSBnZ3Bsb3QoZmVhdHVyZV9kZiwgYWVzKHggPSBsb2dfTWVkaWFuSG91c2VQcmljZSwgeSA9IGxvZ19DZW50cmFsSGVhbGluZ1BlcmNlbnQpKSArDQogIGdlb21fcG9pbnQoKSsNCiAgZ2VvbV9zbW9vdGgobWV0aG9kPWxtKQ0KDQphcGFydG1lbnRzIDwtIGdncGxvdChmZWF0dXJlX2RmLCBhZXMoeCA9IGxvZ19NZWRpYW5Ib3VzZVByaWNlLCB5ID0gbG9nX0ZsYXRBcHJ0KSkgKw0KICBnZW9tX3BvaW50KCkrDQogIGdlb21fc21vb3RoKG1ldGhvZD1sbSkNCg0KY29tZXN0IDwtIGdncGxvdChmZWF0dXJlX2RmLCBhZXMoeCA9IGxvZ19NZWRpYW5Ib3VzZVByaWNlLCB5ID0gbG9nX0NvbUVzdFJlc1RyYW5mb3JtKSkgKw0KICBnZW9tX3BvaW50KCkrDQogIGdlb21fc21vb3RoKG1ldGhvZD1sbSkNCg0KDQoNCmdnYXJyYW5nZShoZWF0aW5nLCBhcGFydG1lbnRzLCBjb21lc3QsIG5jb2w9MywgbnJvdz0xICkNCg0KDQoNCmBgYA0KDQpXaGlsZSB0aGVyZSBkb2VzIGFwcGVhciB0byBiZSBhIHJlbGF0aW9uc2hpcCB3aXRoIHNvbWUgb2YgdGhlc2UsIGl0IGlzbid0IHN0cm9uZyAtIHRoaXMgc3VnZ2VzdHMgdGhlIGZhY3RvcidzIHdlIGhhdmUgaWRlbnRpZmllZCBhcmUgc2lnbmlmaWNhbnQgbm90IGJlY2F1c2Ugb2YgdGhlaXIgYXNzb2NpYXRpb24gd2l0aCBkZXByaXZhdGlvbiBhbmQgcG92ZXJ0eSwgYnV0IGJlY2F1c2Ugb2Ygd2hhdCB0aGV5IG1lYW4gYWJvdXQgdGhlIHNwZWNpZmljIGNoYXJhY3RlcmlzdGljcyBvZiB0aGUgYXJlYS4NCg0KIyMjIFJhbmRvbSBGb3Jlc3RzDQoNClVubGlrZSByZWdyZXNzaW9uLCByYW5kb20gZm9yZXN0IGRvZXNuJ3QgcmVhbGx5IG9uIGFueSBzcGVjaWZpYyB0eXBlIG9mIGFzc29jaWF0aW9uIC0gaW5zdGVhZCwgd2UgcmVseSBvbiBjb21wdXRpbmcgcG93ZXIsIHJlcGV0aXRpb24gYW5kIGl0ZXJhdGlvbiB0byBjYXB0dXJlIHRoZSB2ZXJ5IGJlc3QgcHJlZGljdG9yIGZvciBvdXIgdmFyaWFibGUsIGluIGFueSBjb21iaW5hdGlvbi4gDQoNClRoZXJlIGFyZSByaXNrcyB0byB0aGlzIG1ldGhvZDogb3VyIHNhbXBsZSBzaXplIGlzIHNtYWxsZXIgdGhhbiBJJ2QgbGlrZSwgYW5kIHRoaXMgbWF5IGxlYWQgdG8gb3Zlci1maXQgb2Ygb3V0bGllciBNU09Bcy4NCg0KSXQgZG9lcyBtZWFuIHdlIGRvbid0IG5lZWQgdG8gd29ycnkgYWJvdXQgdHJhbnNmb3JtYXRpb25zIG9yIGNvcnJlbGF0aW9uczogd2UgY2FuIHJldHVybiB0byBvdXIgb3JpZ2luYWwgZGF0YXNldCwgYW5kIGxldCB0aGUgbW9kZWwgaWRlbnRpZnkgdGhlIHN0cm9uZ2VzdCBwcmVkaWN0b3JzLg0KDQpgYGB7cn0NCiMgd2UgcmVtb3ZlIHJvd3Mgd2hlcmUgb3VyIG1haW4gZXJyb3IgaXMgbmEgb3IgbWlzc2luZw0KcmZfbXNvYV9tYXRyaXggPC0gZHJvcF9uYShtc29hX21hdHJpeF9udW1lcmljLCBSUERSb2JiZXJ5U2hpZnRlZCkNCg0KI3RoZW4gd2UgcmVwZWF0IGZvciBhbnkgY29sdW1ucyB3aGVyZSB2YWx1ZSBhc3JlIG1pc3NpbmcNCmNsZWFuX3JmX21hdHJpeCA8LSByZl9tc29hX21hdHJpeFsgLCBjb2xTdW1zKGlzLm5hKHJmX21zb2FfbWF0cml4KSkgPT0gMF0NCg0KDQojdGhlbiB3ZSBkcm9wIG91dCBhbnkgdmFsdWVzIHRoYXQgZGlyZWN0bHkgcHJlZGljdCBvdXIgZXJyb3IuDQoNCmRyb3A8LSBjKCJidXJnbGFyeUFjdHVhbCIsImJ1cmdsYXJ5RXJyb3IiLCJidXJnbGFyeVBlcmNlbnRFcnJvciIsImJ1cmdsYXJ5UHJlZGljdGVkIiwicm9iYmVyeUFjdHVhbCIsInJvYmJlcnlQcmVkaWN0ZWQiLCJyb2JiZXJ5RXJyb3IiLCJyb2JiZXJ5UGVyY2VudEVycm9yIiwgIlJQREJ1cmdsYXJ5U2hpZnRlZCIsICJSUERSb2JiZXJ5U2hpZnRlZCIpDQoNCiNyZW1vdmUgb3V0IHNlbGVjdGVkIGNvbHVtbnMNCmRhdGE8LSBjbGVhbl9yZl9tYXRyaXhbLCEobmFtZXMoY2xlYW5fcmZfbWF0cml4KSAlaW4lIGRyb3ApXQ0KDQoNCiNhdXRvbWF0aWNhbGx5IHJlbW92ZSB3aGl0ZSBzcGFjZSBhbmQgbWFrZSBuYW1lcyByIGNvbXBhdGJpZWwNCm5hbWVzKGNsZWFuX3JmX21hdHJpeCk8LW1ha2UubmFtZXMobmFtZXMoY2xlYW5fcmZfbWF0cml4KSx1bmlxdWUgPSBUUlVFKQ0KDQpkcm9wPC0gYygiYnVyZ2xhcnlBY3R1YWwiLCJidXJnbGFyeUVycm9yIiwicm9iYmVyeUFjdHVhbCIsInJvYmJlcnlFcnJvciIsIlJQREJ1cmdsYXJ5IiwiUlBEUm9iYmVyeSIsInJvYmJlcnlBY3R1YWxTaGlmdGVkIiwicm9iYmVyeVByZWRpY3RlZFNoaWZ0ZWQiLCJidXJnbGFyeUFjdHVhbFNoaWZ0ZWQiLCJidXJnbGFyeVByZWRpY3RlZFNoaWZ0ZWQiLCAiYnVyZ2xhcnlQcmVkaWN0ZWQiLCAiYnVyZ2xhcnlQZXJjZW50RXJyb3IiLCAicm9iYmVyeVByZWRpY3RlZCIsICJyb2JiZXJ5UGVyY2VudEVycm9yIikNCiNkcm9wPC0gYygiYnVyZ2xhcnlQZXJjZW50RXJyb3IiLCJidXJnbGFyeVByZWRpY3RlZCIsInJvYmJlcnlQcmVkaWN0ZWQiLCJyb2JiZXJ5UGVyY2VudEVycm9yIikNCg0KZGF0YTwtIGNsZWFuX3JmX21hdHJpeFssIShuYW1lcyhjbGVhbl9yZl9tYXRyaXgpICVpbiUgZHJvcCldDQoNCg0KbmFtZXMoZGF0YSk8LSBtYWtlLm5hbWVzKG5hbWVzKGRhdGEpLHVuaXF1ZSA9IFRSVUUpDQpkYXRhDQpgYGANCldlIHN0YXJ0IHdpdGggbW9kZWxsaW5nIG91ciByZWxhdGl2ZSByYXRlIG9mIGJ1cmdsYXJ5IHNoaWZ0LiBXZSBkaXZpZGUgb3VyIHNhbXBsZSBpbnRvIGEgdHJhaW5pbmcgc2V0IHdoaWNoIHdlJ2xsIHVzZSB0byB0cmFpbiB0aGUgbW9kZWwsIGFuZCBhIHRlc3Qgc2V0IHdlJ2xsIHVzZSB0byB2ZXJpZnkgYWNjdXJhY3kgYW5kIHZhbGlkaXR5Lg0KDQpgYGB7cn0NCnNhbXBsZSA9IHNhbXBsZS5zcGxpdChkYXRhJFJQRGJ1cmdsYXJ5U2hpZnRlZCwgU3BsaXRSYXRpbyA9IDAuNykNCnRyYWluID0gc3Vic2V0KGRhdGEsIHNhbXBsZSA9PSBUUlVFKQ0KdGVzdCAgPSBzdWJzZXQoZGF0YSwgc2FtcGxlID09IEZBTFNFKQ0KDQoNCnJmX2J1cmdsYXJ5IDwtIHJhbmRvbUZvcmVzdCgNCiAgUlBEYnVyZ2xhcnlTaGlmdGVkIH4gLiwNCiAgZGF0YT10cmFpbiwgDQogIGltcG9ydGFuY2U9VFJVRQ0KKQ0KDQpzdW1tYXJ5KHJmX2J1cmdsYXJ5KQ0KYGBgDQpgYGB7cn0NCiNjYWxjdWxhdGUgb3VyIHByZWRpY3Rpb25zIGFuZCBhICBybXNlDQpwcmVkaWN0aW9uIDwtcHJlZGljdChyZl9idXJnbGFyeSwgdGVzdCkNCk1ldHJpY3M6OnJtc2UodGVzdCRSUERidXJnbGFyeVNoaWZ0ZWQsIHByZWRpY3Rpb24pDQpgYGANCg0KV2hpbGUgbWFjaGluZSBsZWFybmluZyBtb2RlbHMgd2VyZSBvbmNlIGNvbnNpZGVyZWQgb3BhcXVlIGFuZCBkaWZmaWN1bHQgdG8gaW50ZXJwcmV0LCBzZXZlcmFsIGxpYnJhcmllcyBub3cgb2ZmZXIgZnVuY3Rpb25hbGl0eSB0byBleHBsYWluIHByZWRpY3Rpb25zLiBIZXJlIEkgdXNlIFtEQUxFWF0oaHR0cHM6Ly9jcmFuLnItcHJvamVjdC5vcmcvd2ViL3BhY2thZ2VzL0RBTEVYL2luZGV4Lmh0bWwpIHRvIGRvIGp1c3QgdGhpcy4gDQoNCmBgYHtyfQ0KDQpyZl9leHBsYWluZXJfYnVyZ2xhcnkgPC0gREFMRVg6OmV4cGxhaW4ocmZfYnVyZ2xhcnksIGRhdGE9dHJhaW4sIHk9IHRyYWluJFJQRGJ1cmdsYXJ5U2hpZnRlZCkNCg0KcmZfcGVyZl9idXJnIDwtIG1vZGVsX3BlcmZvcm1hbmNlKHJmX2V4cGxhaW5lcl9idXJnbGFyeSkNCnJmX3BlcmZfYnVyZw0KYGBgDQpXZSBjYW4gc2VlIHRoYXQgb3VyIG1vZGVsIHNpZ25pZmljYW50bHkgb3V0cGVyZm9ybXMgb3VyIGJlc3QgbGluZWFyIG1vZGVsczogdGhlIFIyIHN1Z2dlc3RzIHRoYXQgYWxtb3N0IDg1JSBvZiB0aGUgdmFyaWFuY2UgaXMgY29ycmVjdGx5IGludGVycHJldGVkLCBhbmQgb3VyIHJtc2UgKHJvb3QgbWVhbiBzcXVhcmVkIGVycm9yKSBpcyBhcm91bmQgMC4yIG9uIG91ciB0ZXN0IHNldC4NCg0KVGhpcyBtb2RlbCB3b3VsZCBiZSBpbGwtc3VpdGVkIHRvIHByZWRpY3Rpb24gb3Igb3BlcmF0aW9uYWxpc2luZyAtaXQgaXMgYSBkZWZhdWx0IGZvcmVjYXN0IHdpdGggbm8gaHlwZXItcGFyYW1ldGVyIHR1bmluZywgYW5kIG5vIGNvbnNpZGVyYXRpb24gb2YgZXJyb3IgcmF0ZXMgLSBidXQgdXNpbmcgdG9vbHMgbGlrZSBEQUxFWCwgd2UgY2FuIGlkZW50aWZ5IHdoaWNoIHByZWRpY3RvcnMgdGhlIG1vZGVsIGlkZW50aWZpZXMgYXMgbW9zdCBpbXBvcnRhbnQuIA0KDQpgYGB7ciwgZmlnLndpZHRoID0gMTN9DQptb2RlbF9wYXJ0c19idXJnIDwtbW9kZWxfcGFydHMocmZfZXhwbGFpbmVyX2J1cmdsYXJ5KQ0KbW9kZWxfcGFydHNfYnVyZw0KDQoNCmBgYA0KDQpgYGB7ciwgZmlnLndpZHRoID0gMTN9DQpwbG90KG1vZGVsX3BhcnRzX2J1cmcsIG1heF92YXJzPTI1KQ0KDQpgYGANClRoZSB0aHJlZSBtb3N0IGltcG9ydGFudCBmZWF0dXJlcyBvdXIgbW9kZWwgaGlnaGxpZ2h0cyBhcmU6DQotIHRoZSBhZ2UgY29tcG9zaXRpb24gb2YgdGhlIHBvcHVsYXRpb24NCi0gdGhlIG51bWJlciBvZiByZXNpZGVudHMgd2hpY2ggYXJlIGluIGNvbW1lcmNpYWwgcHJvcGVydHkNCi0gdGhlIGhpc3RvcmljIG51bWJlciBvZiBidXJnbGFyaWVzDQoNClRoaXMgc2VlbXMgdG8gY29ycm9ib3JhdGUgb3VyIHByZXZpb3VzIHJlZ3Jlc3Npb24gbW9kZWwuIFRvIGZ1cnRoZXIgdW5waWNrICB0aGVzZSB0cmVuZHMsIHdlIGNhbiB1c2UgUGFydGlhbCBEZXBlbmRlbmNlIFBsb3RzIHRvIGlkZW50aWZ5IGhvdyB0aGUgbW9kZWwgcHJlZGljdGlvbiBzaGlmdHMgd2l0aCB0aGVzZSB2YWx1ZXMuIA0KDQpgYGB7cn0NCnBkcCA8LSBtb2RlbF9wcm9maWxlKHJmX2V4cGxhaW5lcl9idXJnbGFyeSkNCg0KYGBgDQoNCmBgYHtyfQ0KcGxvdChwZHAsIHZhcmlhYmxlcz0idG90YWxfYnVyZ2xhcmllcyIpDQoNCmBgYA0KV2Ugc3RpbGwgc2VlIGEgc3Ryb25nIGFzc29jaWF0aW9uIGJldHdlZW4gYSBoaWdoIG51bWJlciBvZiBoaXN0b3JpYywgYW5kIGEgbGFyZ2UgcmVkdWN0aW9uIGR1cmluZyB0aGUgbG9ja2Rvd24gcGVyaW9kDQoNCmBgYHtyfQ0KcGxvdChwZHAsIHZhcmlhYmxlcz0gIk1pZC55ZWFyLkVzdGltYXRlcy4yMDEyLi5ieS5hZ2UuMC40IikNCg0KYGBgDQpNU09BcyB3aXRoIHZlcnkgbGFyZ2UgKnZlcnkgeW91bmcqIHBvcHVsYXRpb25zIHNlZW0gdG8gaGF2ZSBhIGxlc3Mgc3Ryb25nIGJ1cmdsYXJ5IGxvY2tkb3duIGRlY3JlYXNlICh0aG91Z2ggdGhpcyBpcyBsaWtlbHkgaGVhdmlseSBhc3NvY2lhdGVkIHRvIGRlcHJpdmF0aW9uKQ0KDQpgYGB7cn0NCnBsb3QocGRwLCB2YXJpYWJsZXM9IkNvbUVzdFJlcyIpDQpgYGANCmBgYHtyfQ0KaGlzdChtc29hX21hdHJpeF9udW1lcmljJENvbUVzdFJlcykNCmBgYA0KQnkgY29tYmluaW5nIHRoZSBkaXN0cmlidXRpb24gb2YgY29tbWVyY2lhbCByZXNpZGVudHMgYnkgTVNPQSB3aXRoIG91ciBQRFAsIHdlIGNhbiBzZWUgdGhhdCB0aG9zZSBNU09BcyB0aGF0IGFyZSBtb3N0IGRlbnNlbHkgcG9wdWxhdGVkIGJ5IGNvbW1lcmNpYWwgcmVzaWRlbnRzIHNlZSB0aGUgc21hbGxlc3QgImNvdmlkIGRlY3JlYXNlIihzdWdnZXN0aW5nIHRoYXQgdGhvc2UgTVNPQXMgdGhhdCBhcmUgdmVyeSBoZWF2aWx5IHJlc2lkZW50aWFsIHNhdyB0aGUgc2hhcnBlc3QgZHJvcHMpLg0KDQpgYGB7cn0NCnBsb3QocGRwLCB2YXJpYWJsZXM9IkhvdXNlLlByaWNlcy5TYWxlcy4yMDA4IikNCg0KYGBgDQoNCkZpbmFsbHksIHdlIGNhbiBzZWUgdGhhdCB0aG9zZSBhcmVhcyB0aGF0IGV4cGVyaWVuY2VkIHRoZSBoaWdoZXN0IHZvbHVtZSBvZiBob3VzZSBzYWxlcyBleHBlcmllbmNlZCB0aGUgbG93ZXN0IHJlbGF0aXZlIGRlY3JlYXNlIGluIGJ1cmdsYXJ5Lg0KDQpUaGlzIGhpZ2hsaWdodHMgdGhlIGltcG9ydGFuY2Ugb2YgaWRlbnRpZnlpbmcgY29ycmVsYXRlcyBpbiBSRiBtb2RlbHMgLSBlc3BlY2lhbGx5IGluIGEgZGF0YXNldCB3aGVyZSBmZWF0dXJlcyBhcmUgaGlnaGx5IGludGVybGlua2VkLCBhc3NvY2lhdGlvbiBkb2VzIG5vdCBpbXBseSBjYXVzYXRpb24sIGFuZCB3ZSBzaG91bGQgYmUgd2FyeSBvZiBvdmVyLWludGVycHJldGluZy4NCg0KV2UgY2FuIG5vdyByZXBlYXQgb3VyIHByb2Nlc3MgZm9yIG91ciByb2JiZXJ5IHNoaWZ0Lg0KDQoNCmBgYHtyfQ0KDQpzYW1wbGUgPSBzYW1wbGUuc3BsaXQoZGF0YSRSUERSb2JiZXJ5U2hpZnRlZCwgU3BsaXRSYXRpbyA9IDAuNzUpDQp0cmFpbiA9IHN1YnNldChkYXRhLCBzYW1wbGUgPT0gVFJVRSkNCnRlc3QgID0gc3Vic2V0KGRhdGEsIHNhbXBsZSA9PSBGQUxTRSkNCg0KDQpyZl9yb2JiZXJ5IDwtIHJhbmRvbUZvcmVzdCgNCiAgUlBEUm9iYmVyeVNoaWZ0ZWQgfiAuLA0KICBkYXRhPXRyYWluLCANCiAgaW1wb3J0YW5jZT1UUlVFDQopDQoNCnN1bW1hcnkocmZfcm9iYmVyeSkNCg0KYGBgDQpXZSd2ZSBub3cgdHJhaW5lZCBhIG1vZGVsLiAgTGV0J3Mgbm93IHVzZSB0aGUgREFMRVggbGlicmFyeSB0byB1bmRlcnN0YW5kIGl0LCBhbmQgc2VlIGhvdyBpdCBwZXJmb3Jtcy4NCg0KYGBge3J9DQoNCnJmX2V4cGxhaW5lcl9yb2JiZXJ5IDwtIERBTEVYOjpleHBsYWluKHJmX3JvYmJlcnksIGRhdGE9dHJhaW4sIHk9IHRyYWluJFJQRFJvYmJlcnlTaGlmdGVkKQ0KDQpyZl9wZXJmX3JvYiA8LSBtb2RlbF9wZXJmb3JtYW5jZShyZl9leHBsYWluZXJfcm9iYmVyeSkNCnJmX3BlcmZfcm9iDQpgYGANCg0KDQoNCg0KYGBge3IsIGZpZy53aWR0aCA9IDEzfQ0KbW9kZWxfcGFydHNfcm9iIDwtbW9kZWxfcGFydHMocmZfZXhwbGFpbmVyX3JvYmJlcnkpDQptb2RlbF9wYXJ0c19yb2INCmBgYA0KDQpgYGB7ciwgZmlnLndpZHRoID0gMTN9DQpwbG90KG1vZGVsX3BhcnRzX3JvYiwgbWF4X3ZhcnM9MjUpDQpgYGANCg0KYGBge3J9DQpwZHBfcm9iIDwtIG1vZGVsX3Byb2ZpbGUocmZfZXhwbGFpbmVyX3JvYmJlcnkpDQpgYGANCg0KYGBge3J9DQpwbG90KHBkcF9yb2IsIHZhcmlhYmxlcz0idG90YWxfcm9iYmVyaWVzIikNCg0KYGBgDQoNCmBgYHtyfQ0KcGxvdChwZHBfcm9iLCB2YXJpYWJsZXM9InRvdGFsX2J1cmdsYXJpZXMiKQ0KDQpgYGANCg0KVGhlIGVmZmVjdCBvZiBoaXN0b3JpYyBjcmltZSBlZmZlY3RzIGFnYWluIGFwcGVhcnMgbGlrZSBhIHJlbGlhYmxlIHByZWRpY3RvciBvZiBhIHJvYmJlcnkgY292aWQgc2hpZnQ6IHRob3NlIE1TT0FzIHdpdGggdGhlIGhpZ2hlc3QgbnVtYmVyIG9mIGJ1cmdsYXJpZXMgYW5kIHJvYmJlcmllcyBzZWUgc3Ryb25nIGRlY3JlYXNlcyBpbiByb2JiZXJ5ICh0aG91Z2ggdGhlIGFzc29jaWF0aW9uIHdpdGggYnVyZ2xhcnkgaXMgbm90IGNsZWFyIGN1dCwgc3VnZ2VzdGluZyBvdGhlciBpbnRlcmFjdGlvbiBlZmZlY3RzIG1heSBiZSBkcml2aW5nIHRoaXMpDQoNCmBgYHtyfQ0KcGxvdChwZHBfcm9iLCB2YXJpYWJsZXM9IlJvYWQuQ2FzdWFsdGllcy4yMDExLjIwMTEuVG90YWwiKQ0KDQpgYGANClJvYWQgY2FzdWFsdGllcyBpcyBhbm90aGVyIHN0cm9uZyByZWxhdGlvbnNoaXAuIFRoaXMgY291bGQgYmUgYSBwcm94eSBmb3IgZGVwcml2YXRpb24sIGJ1dCBJIHN1Z2dlc3QgdGhpcyBpcyBtb3JlIGRvd24gdG8gZ2VvZ3JhcGhpYyBmZWF0dXJlcyAtIHJvYWQgY2FzdWFsdGllcyBhcmUgcHJvYmFibHkgcmFyZXIgaW4gZGVuc2VyIHVyYmFuIGVudmlyb25tZW50cywgbW9zdCBsaWtlbHkgdG8gYmUgYWZmZWN0ZWQgYnkgbG9ja2Rvd24uDQoNCmBgYHtyfQ0KcGxvdChwZHBfcm9iLCB2YXJpYWJsZXM9IkV0aG5pYy5Hcm91cC4uMjAxMS5DZW5zdXMuLk90aGVyLmV0aG5pYy5ncm91cC4uLi4iKQ0KYGBgDQpgYGB7cn0NCmhpc3QoZGF0YSRFdGhuaWMuR3JvdXAuLjIwMTEuQ2Vuc3VzLi5PdGhlci5ldGhuaWMuZ3JvdXAuLi4uKQ0KDQpgYGANCg0KDQpGaW5hbGx5LCB3ZSBzZWUgYSBkZW1vZ3JhcGhpYyBwcmVkaWN0b3IgbGlua2VkIHRvICJvdGhlciBldGhuaWMgZ3JvdXAiLiBJJ20gbm90IGNsZWFyIGhvdyB0byBpbnRlcnByZXQgdGhpcywgYnV0IGl0IHN1Z2dlc3RzIHRoYXQgdGhvc2UgTVNPQXMgdGhhdCBhcmUgbW9zdCBkaXZlcnNlIChhbmQgaGF2ZSB0aGUgbGFyZ2VzdCByZXByZXNlbnRhdGlvbiBieSB0aGVzZSBldGhuaWMgZ3JvdXBzKSBleHBlcmllbmNlZCB0aGUgc3Ryb25nZXN0IGRlY3JlYXNlcyBpbiByb2JiZXJ5IGR1cmluZyBsb2NrZG93bi4gDQoNClRvZ2V0aGVyLCB0aGVzZSBhbmFseXNlcyBzdWdnZXN0IHRoYXQgdGhlIGNyaW1lIGRyb3AgZm9yIHJvYmJlcnkgYW5kIGJ1cmdsYXJ5IGR1cmluZyBuYXRpb25hbCBsb2NrZG93biB3YXMgc2lnbmlmaWNhbnRseSBhZmZlY3RlZCBieSBkaXN0aW5jdCBsb2NhbCBmYWN0b3JzLiBGb3IgYm90aCBvZmZlbmNlcywgaGlzdG9yaWNhbCBjcmltZSB0cmVuZHMgcGxheSBhIHJvbGUsIHdpdGggaGlnaCBjcmltZSBhcmVhcyBleHBlcmllbmNpbmcgYSByZWxhdGl2ZWx5IGxhcmdlciBkcm9wIGluIGJvdGggYnVyZ2xhcnkgYW5kIHJvYmJlcnkuDQoNCkJleW9uZCB0aGF0LCB0aGUgZHJpdmVycyB2YXJ5IGZvciBlYWNoIG9mZmVuY2UgIHR5cGU6IGJ1cmdsYXJ5IHdhcyBkcml2ZW4gYnkgdGhlIGNvbXBvc2l0aW9uIG9mIHRoZSByZXNpZGVudGlhbCBwb3B1bGF0aW9uLCB3aXRoIGhlYXZpbHkgcmVzaWRlbnRpYWwgYXJlYXMsIGFuZCBhcmVhcyB3aXRoIGEgcmVsYXRpdmVseSAic3RhYmxlIiBwb3B1bGF0aW9uIGFzIG1lYXN1cmVkIGJ5IGxvdyBob3VzaW5nIHNhbGVzIGFsc28gc2F3IGxhcmdlciBkZWNyZWFzZSAtIHRoaXMgaXMgbGlrZWx5IGR1ZSB0byB0aGUgaW5jcmVhc2VkIG51bWJlciBvZiBlbXB0eSByZXNpZGVudGlhbCBwcm9wZXJ0aWVzLg0KDQpSb2JiZXJ5IGRlY3JlYXNlcyBjb252ZXJzZWx5LCBhcmUgd2VsbCBhc3NvY2lhdGVkIHdpdGggaGlnaCBudW1iZXJzIG9mIHJvYWQgY2FzdWFsdGllcywgYXMgd2VsbCBhcyB0aGUgZXRobmljIG1ha2V1cCBvZiB0aGUgbG9jYWwgcG9wdWxhdGlvbiAtIHRoaXMgaXMgbGlrZWx5IGR1ZSB0byBhbiBhc3NvY2lhdGlvbiB3aXRoIGRlbnNlciwgbW9yZSB1cmJhbiBhcmVhcywgd2l0aCBsb3dlciBzdHJlZXQgc3BlZWRzLCBhbmQgYSBwb3NzaWJsZSBsaW5rIHRvIGRlcHJpdmF0aW9uLCB3aGVyZWJ5IG1pbm9yaXR5IHBvcHVsYXRpb24gd2VyZSBsZWFzdCBhYmxlIHRvIHdvcmsgZnJvbSBob21lLCBhbmQgd2VyZSBsaWtlbHkgdG8gc3RpbGwgcHJlc2VudCBhcyBhdmFpbGFibGUgdGFyZ2V0cyBmb3Igcm9iYmVyeS4NCg==